{"id":1908,"date":"2019-04-17T12:00:00","date_gmt":"2019-04-17T10:00:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/machine-learning-101-the-bayes-classfier\/"},"modified":"2026-07-05T03:22:58","modified_gmt":"2026-07-05T01:22:58","slug":"machine-learning-101-the-bayes-classfier","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/machine-learning-101-the-bayes-classfier\/","title":{"rendered":"Machine Learning 101 \u2013 The Bayes\u2019 Classfier"},"content":{"rendered":"

This is the second lecture on classification. It follow the first one: Introduction to Classification<\/a>.<\/p>\n

Bayes’ Classifier<\/strong> is a classifier that works based on Bayes’ Theorem. It assigns each observation to the most likely class given the values of the measurements.<\/p>\n

Also remember that Bayes’ theorem helps us compute conditional probabilities. So a Bayes’ classifier determines the conditional probability for the particular class given the features.<\/p>\n

 <\/p>\n

Let’s take the X-ray example.<\/em><\/p>\n

When an X-ray image of the patient is obtained, the objective is to decide which of the two classes to assign it.\u00a0 Either C1<\/sub> or C2<\/sub>.<\/p>\n

There for we need to compute the conditional probabilities of the classes C1<\/sub> and C2<\/sub> given the image.\u00a0 Let’s say Ck<\/sub>, where k=1, 2.<\/p>\n

The conditional probabilities is given by:<\/p>\n

 <\/p>\n

\"Bayes'<\/p>\n

 <\/p>\n

In this equation:<\/p>\n

P(Ck<\/sub>) is the prior probability for the class Ck<\/sub>. This is because the probability is know prior to taking the X-ray.<\/p>\n

P(Ck<\/sub> | x<\/strong>) is the corresponding posterior probability. This is the probability of Ck<\/sub> after taking the X-ray. That is after x<\/strong> has been determined.<\/p>\n

If the objective is to reduce the chance of assigning x to the wrong class, then we would choose the class that have the higher posterior probability.<\/p>\n

In the case of two-class problem we just discussed, then Bayes’ classifier corresponds to trying to predict\u00a0 has P(Ck<\/sub> | x<\/strong>) > 0.5. Therefore the corresponding posterior probability would be < 0.5<\/p>\n

Since we know that P(C1<\/sub> | x<\/strong>) + P(C2<\/sub> | x<\/strong>) = 1.<\/p>\n

 <\/p>\n

Bayes’ classifier is know to do pretty good job in classification. However, there are times when misclassification occurs.<\/p>\n

We would end this lesson here so in the next lesson, we discuss how to minimize misclassification rate in Bayes’ classifier.<\/p>\n

 <\/p>\n

You could read up Basics of Conditional Probability<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

This is the second lecture on classification. It follow the first one: Introduction to Classification. Bayes’ Classifier is a classifier that works based on Bayes’ … <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"pagelayer_contact_templates":[],"_pagelayer_content":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-1908","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1908","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=1908"}],"version-history":[{"count":1,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1908\/revisions"}],"predecessor-version":[{"id":2076,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1908\/revisions\/2076"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=1908"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=1908"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=1908"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}