{"id":175,"date":"2018-01-21T18:39:00","date_gmt":"2018-01-21T17:39:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2018\/01\/21\/understanding-clustering-k-means-in-machine-learning-unsupervised-learning\/"},"modified":"2020-08-22T09:21:36","modified_gmt":"2020-08-22T07:21:36","slug":"understanding-clustering-k-means-in-machine-learning-unsupervised-learning","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/understanding-clustering-k-means-in-machine-learning-unsupervised-learning\/","title":{"rendered":"Understanding Clustering (k-Means) in Machine Learning (Unsupervised Learning)"},"content":{"rendered":"
\n
In this lesson we are going discuss clustering under the following topic:<\/p>\n
    \n
  1. Introduction to Clustering<\/span><\/li>\n

    <\/span><\/p>\n

  2. Formalized Definition of k-Means Clustering<\/span><\/li>\n

    <\/span><\/p>\n

  3. 1-of-K Coding Scheme<\/span><\/li>\n

    <\/span><\/p>\n

  4. The Expectation and Maximization Algorithm  <\/span><\/li>\n<\/ol>\n
    <\/a><\/div>\n

    Introduction to Clustering<\/b>
    Clustering is a class of unsupervised learning concept or machine learning whose goal is to identify groups or clusters within datapoints in a multi-dimensional space.
    Suppose we have data sets {x1<\/sub>, x2<\/sub>, …, xn<\/sub>}<\/i><\/span> consisting of N observations of a random D-dimensional Euclidean variable x. The goal is to partition the data into K clusters when the value of K is preknown.
    We can define a cluster as a group of data points whose inter-point distances are minimal compared with the distance with points that are outside the cluster.<\/p>\n

    Formalization of k-Means Clustering<\/b>
    Let’s assume a set of D-dimensional vectors  \u03bck<\/sub> where k = 1, 2, …, K and \u03bck<\/sub> <\/span>represents the centroid of the cluster k. Take centroid to be the mean of data points in the cluster.
    Our goal is to find the assignment  of data points to clusters as well as a set of vectors { \u03bck<\/sub>}<\/span> such that the sum of squares of distances of each data point to its closest vector  \u03bck<\/sub>  is at a minimum.<\/p>\n

    1-of-K Coding Scheme<\/b>
    The 1-of-k Coding Scheme is a coding scheme applied to a K-dimensional vector x in which one of the elements xk<\/sub> equals 1 and all the remaining elements 0. This means that a data point cannot belong to two clusters at the same time.<\/p>\n

    In this case, for each data point x, we introduce a corresponding set of binary indicator variables rnk<\/sub> \u2208 {0,1}.<\/i><\/span><\/span>
    If xn<\/sub> is assigned to \u03bck<\/sub> then rnk<\/sub> = 1<\/span><\/i> and rnj<\/sub> = 0 for j \u2260 k
    We can then define an objective function J know as distortion measure<\/p>\n

    <\/a><\/div>\n

    This represents the sum of the squares of the distances of each of the data points from its assigned vector \u03bck<\/sub><\/span>
     The goal is to choose values for {rnk<\/sub>}<\/span> and {\u03bck<\/sub>}<\/span> so as to minimize J. This brings us to the next section where we would examine the algorithm.<\/p>\n

    The Expectation and Maximization Algorithm(k-Means)<\/b>
     The algorithm is an iterative algorithm where for each iteration, two steps are taken that optimizes the variables rnk<\/sub> <\/span><\/span><\/i>and \u03bck<\/sub><\/p>\n

    The algorithm goes this way. <\/i><\/span>
    First, choose an initial value for \u03bck<\/sub> 
    In the first phase: we minimize J with respect to rnk<\/sub><\/span><\/i> while keeping \u03bck<\/sub><\/span><\/i> fixed.
    In the second phase, we minimize J with respect to \u03bck<\/sub>  while keeping rnk<\/sub><\/span><\/i>  fixed.
    Repeat the two steps until convergence.<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

    In this lesson we are going discuss clustering under the following topic: Introduction to Clustering Formalized Definition of k-Means Clustering 1-of-K Coding Scheme The Expectation … <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[15,11,16],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/175"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=175"}],"version-history":[{"count":1,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/175\/revisions"}],"predecessor-version":[{"id":1440,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/175\/revisions\/1440"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=175"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=175"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=175"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}