{"id":1905,"date":"2019-04-15T12:00:00","date_gmt":"2019-04-15T10:00:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/machine-learning-101-what-is-probability-density\/"},"modified":"2026-07-05T03:22:51","modified_gmt":"2026-07-05T01:22:51","slug":"machine-learning-101-what-is-probability-density","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/machine-learning-101-what-is-probability-density\/","title":{"rendered":"Machine Learning 101 \u2013 What is Probability Density?"},"content":{"rendered":"<p>By now, you probably understand probability as well as probability theory. You also know about the Sum Rule and Product Rule. Then\u00a0 you also understand Bayes&#8217; theorem from <a href=\"https:\/\/kindsonthegenius.com\/tempsite\/machine-learning-101-rules-of-probability-bayes-theorem\/\">Lesson 9<\/a> and\u00a0<a href=\"https:\/\/kindsonthegenius.com\/tempsite\/machine-learning-101-application-of-bayes-theorem\/\"> Lesson 10<\/a>. However, you also need to understand the term Probability Density.<\/p>\n<p>To understand Probability Density, you need to understand the following:<\/p>\n<ul>\n<li>difference between continuous and discrete variable<\/li>\n<li>what is a random variable<\/li>\n<li>probability of a random variable taking on a given value<\/li>\n<\/ul>\n<p>Let&#8217;s start with the first one because we mentioned it when we in <a href=\"https:\/\/kindsonthegenius.com\/tempsite\/machine-learning-101-classes-of-machine-learning-problems\/\">Lecture 3<\/a> under difference between Classification and Regression.<\/p>\n<p>&nbsp;<\/p>\n<p><strong><em>difference between continuous and discrete variable<\/em><\/strong><\/p>\n<p>A discrete variable is one that can take one finite range of values. For example integers between 1 and 9. In this case they are 1, 2, 3, 4, 5, 6, 7, 8 and 9. Another example is result of a cancer test. It could be one of two values: positive or negative.<\/p>\n<p>On the other hand, a continuous variable can take on infinite range of values. For example, real numbers between 1 and 0. In this case it could be 1, 1.01, 1.004, 1.333, 2.32, &#8230;. and so on. The combination is infinite.<\/p>\n<p>You can therefore recall that in classification, we are trying to predict the value of a discreet variable but in regression, we are trying to predict value of a continuous variable.<\/p>\n<p>&nbsp;<\/p>\n<p><strong><em><span style=\"font-size: 1rem;\">what is a random variable<\/span><\/em><\/strong><\/p>\n<p>A random variable is a variable whose possible values are from the outcome of a random experiment.\u00a0 For example, in the experiment of choosing a fruit from a box containing oranges and apples. If the outcome of choosing a fruit is F, then F can be either apple(a) or orange (o).<\/p>\n<p>So we can write it as <em>F = a<\/em> or <em>F = o<\/em>. Here F is\u00a0 a random variable, while <em>o<\/em> or <em>a<\/em> are values of the random variable.<\/p>\n<p>&nbsp;<\/p>\n<p><strong><em>probability of a random variable taking on a given value<\/em><\/strong><\/p>\n<p>This is kind of self-explanatory.\u00a0 So using the example of box of fruits,\u00a0 we can represent the probability of a random variable\u00a0 as:<\/p>\n<p>P(F=a): this means the probability of the random variable F taking the value a. This is normally written as P(a)<\/p>\n<p>P(F = o): this means the probability of the random variable taking the value o. Written as P(o)<\/p>\n<p>&nbsp;<\/p>\n<h4><strong>Probability Density<\/strong><\/h4>\n<p>The probability density is the probability that a random variable takes on a value between two values.<\/p>\n<p>Now, we can apply this to continuous variables. Consider a continuous random variable x.<\/p>\n<p>Let p(x) be the probability that x takes certain values. This function is called the <em>probability density function.<\/em> Then the values of p(x) can range from 0 to 1.<\/p>\n<p>Also, x can take on any value from 0 to infinity.<\/p>\n<p>Now, we can try to calculate the probability that the random variable x takes a value between a and b.\u00a0 The result will be the sum of all the probabilities for values from <em>a<\/em> to <em>b.<\/em> We call this the <em>probability density<\/em> over <em>x<\/em>.<\/p>\n<p>This sum can be written in integral form as:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-726 aligncenter\" src=\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/Probability-density-300x77.jpg\" alt=\"Probability density\" width=\"257\" height=\"66\" \/><\/p>\n<p>More formally we say that, if the probability of a random variable x falling in the interval<em> (x,\u00a0 x +\u00a0\u03b4x)<\/em> is given by\u00a0<em> p(x)\u03b4x<\/em>\u00a0 \u00a0 for\u00a0 \u00a0\u00a0<em>\u03b4x\u00a0\u2192 0<\/em>,\u00a0 \u00a0then p(x) is the probability density over <em>x<\/em>.<\/p>\n<p>I you know a bit of calculus, then you will recognize that the probability density is the same as the shaded area in Figure 1 below.<\/p>\n<p>&nbsp;<\/p>\n<figure id=\"attachment_727\" aria-describedby=\"caption-attachment-727\" style=\"width: 418px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-727\" src=\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/What-is-Probability-Density.jpg\" alt=\"What is Probability Density\" width=\"418\" height=\"318\" \/><figcaption id=\"caption-attachment-727\" class=\"wp-caption-text\">Figure 1: Probability Density <em>p(x)<\/em> over <em>x<\/em><\/figcaption><\/figure>\n<p>&nbsp;<\/p>\n<p><strong>Final Notes<\/strong><\/p>\n<p>This is the much we&#8217;ll taken on Probability Density and Probability Density Function.<\/p>\n<p>Although there are a few other terms but I&#8217;ll rather say you don&#8217;t worry about them for now. For instance, cumulative distribution function and probability mass function<\/p>\n","protected":false},"excerpt":{"rendered":"<p>By now, you probably understand probability as well as probability theory. You also know about the Sum Rule and Product Rule. Then\u00a0 you also understand &hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"pagelayer_contact_templates":[],"_pagelayer_content":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-1905","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1905","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=1905"}],"version-history":[{"count":1,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1905\/revisions"}],"predecessor-version":[{"id":2073,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1905\/revisions\/2073"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=1905"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=1905"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=1905"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}