{"id":226,"date":"2017-12-17T03:01:00","date_gmt":"2017-12-17T02:01:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2017\/12\/17\/dimensionality-reduction-and-principal-component-analysis-pca\/"},"modified":"2020-08-22T14:47:40","modified_gmt":"2020-08-22T12:47:40","slug":"dimensionality-reduction-and-principal-component-analysis-pca","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/dimensionality-reduction-and-principal-component-analysis-pca\/","title":{"rendered":"Dimensionality Reduction and Principal Component Analysis (PCA)"},"content":{"rendered":"<div style=\"color: #555555; font-size: 18px; line-height: 30px; text-align: justify;\">\n<div style=\"font-family: 'segoe ui';\">\n<div style=\"text-align: center;\"><a href=\"https:\/\/youtu.be\/z2rMJENiHsI\" target=\"_blank\">More Detailed Video Explanation Here <\/a><br \/><a href=\"https:\/\/youtu.be\/No-IMRjzVKY\" target=\"_blank\">Video on How to Perform PCA in R here <\/a><\/div>\n<p>We would explain the concept of dimensionality reduction in a very simple way. In line with the principle of my articles, I would try to be as clear as possible.<br \/>In this lesson, we would focus on the explaining the concept and in the next lesson, we would look at the underlying derivation of the technique.<br \/>&nbsp;<b>Content<\/b> <\/p>\n<ul>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Problem with High Dimensional Data<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">What is Dimensionality Reduction<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Two Type of Dimensionality Reduction<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">What is Principal Components Analysis<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Methods of Dimensionality Reduction<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">How PCA Works<\/span><\/span><\/span><\/li>\n<ul>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Demonstration of PCA <\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Obtain Covariance Matrix<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Obtain Eigen Pairs<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Obtain Scores and Loadings<\/span><\/span><\/span><\/li>\n<\/ul>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Extract the Principal Components<\/span><\/span><\/span><\/li>\n<li><span style=\"font-size: small;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"color: black;\">Summary<\/span><\/span><\/span><\/li>\n<\/ul>\n<div style=\"clear: both; text-align: center;\"><a href=\"https:\/\/3.bp.blogspot.com\/-aGt6JrHLt_E\/WjXd4Ge7B-I\/AAAAAAAAAgY\/-gF8sXb4qqAwObOmP9hTiJa0ot5wR82ZACLcBGAs\/s1600\/Principal-Component-Analysis-And-Dimensionality-Reduction.jpg\" style=\"margin-left: 1em; margin-right: 1em;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"737\" data-original-width=\"1452\" height=\"162\" src=\"https:\/\/3.bp.blogspot.com\/-aGt6JrHLt_E\/WjXd4Ge7B-I\/AAAAAAAAAgY\/-gF8sXb4qqAwObOmP9hTiJa0ot5wR82ZACLcBGAs\/s320\/Principal-Component-Analysis-And-Dimensionality-Reduction.jpg\" width=\"320\" \/><\/a><\/div>\n<p><b>The Simple Explanation<\/b><br \/>You already know that if you are given data in two dimension, say x and y, you could probably plot the graph and see the relationship. What if you are given data in three dimension? You could still try to create the plot but if the data is large enough, visualizing the plot would be difficult.<br \/>Now what if the data is in 10 or 20 or even 100&nbsp; and more? How could you plot it? Even if you could, you find out that it may not make much sense. This is were dimensionality reduction or dimensional reduction comes in.<\/p>\n<p><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"8227894917\" style=\"display: block; text-align: center;\"><\/ins> <b>Formal Definition of Dimensionality Reduction<\/b><br \/>&#8220;&#8230;is the process of reducing the number of random variables under consideration by obtaining a set of principal variables&#8221; &#8211; <i>Wikipedia<\/i><br \/>&#8220;&#8230; it the process of reducing the number of variables or features in review&#8221; &#8211; <i>Big Data University<\/i><\/p>\n<p><b>Problem of High-Dimensional Data<\/b><\/p>\n<ul>\n<li>training a model with high-dimensional data requires much time-space complexity<\/li>\n<li>Overfitting<\/li>\n<li>Not all the features of the data are relevant to the problem being solved<\/li>\n<li>Data in lower dimension has lower noise(unnecessary parts of the data)<\/li>\n<\/ul>\n<p><b>Type of Dimensionality Reduction<\/b><br \/>The two types of dimensionality reduction are:<br \/><b>1. Feature Extraction:<\/b> This technique has to do with finding new features in the data after it has been transformed from a high-dimensional space to a low dimensional space.<\/p>\n<p><b>2. Feature Selection<\/b>: This have to do with finding the most relevant features to a problem. This is done by obtaining a subset or key features of the original variables<\/p>\n<p><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"8227894917\" style=\"display: block; text-align: center;\"><\/ins> <\/p>\n<div style=\"text-align: justify;\"><b>Methods of Dimesionality Reduction<\/b><\/div>\n<div style=\"text-align: justify;\"><!--[if gte mso 9]><xml> <o:OfficeDocumentSettings>  <o:AllowPNG\/> <\/o:OfficeDocumentSettings><\/xml><![endif]--><\/div>\n<p><!--[if gte mso 9]><xml> <w:WordDocument>  <w:View>Normal<\/w:View>  <w:Zoom>0<\/w:Zoom>  <w:TrackMoves\/>  <w:TrackFormatting\/>  <w:PunctuationKerning\/>  <w:ValidateAgainstSchemas\/>  <w:SaveIfXMLInval>false<\/w:SaveIfXMLInvalid>  <w:IgnoreMixedContent>false<\/w:IgnoreMixedContent>  <w:AlwaysShowPlaceholderText>false<\/w:AlwaysShowPlaceholderText>  <w:DoNotPromoteQF\/>  <w:LidThemeOther>EN-GB<\/w:LidThemeOther>  <w:LidThemeAsian>X-NONE<\/w:LidThemeAsian>  <w:LidThemeComplexScript>X-NONE<\/w:LidThemeComplexScript>  <w:Compatibility>   <w:BreakWrappedTables\/>   <w:SnapToGridInCell\/>   <w:WrapTextWithPunct\/>   <w:UseAsianBreakRules\/>   <w:DontGrowAutofit\/>   <w:SplitPgBreakAndParaMark\/>   <w:EnableOpenTypeKerning\/>   <w:DontFlipMirrorIndents\/>   <w:OverrideTableStyleHps\/>  <\/w:Compatibility>  <m:mathPr>   <m:mathFont m:val=\"Cambria Math\"\/>   <m:brkBin m:val=\"before\"\/>   <m:brkBinSub m:val=\"&#45;-\"\/>   <m:smallFrac m:val=\"off\"\/>   <m:dispDef\/>   <m:lMargin m:val=\"0\"\/>   <m:rMargin m:val=\"0\"\/>   <m:defJc m:val=\"centerGroup\"\/>   <m:wrapIndent m:val=\"1440\"\/>   <m:intLim m:val=\"subSup\"\/>   <m:naryLim m:val=\"undOvr\"\/>  <\/m:mathPr><\/w:WordDocument><\/xml><![endif]--><!--[if gte mso 9]><xml> <w:LatentStyles DefLockedState=\"false\" DefUnhideWhenUsed=\"true\"   DefSemiHidden=\"true\" DefQFormat=\"false\" DefPriority=\"99\"   LatentStyleCount=\"267\">  <w:LsdException Locked=\"false\" Priority=\"0\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Normal\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"heading 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 7\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 8\"\/>  <w:LsdException Locked=\"false\" Priority=\"9\" QFormat=\"true\" Name=\"heading 9\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 7\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 8\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" Name=\"toc 9\"\/>  <w:LsdException Locked=\"false\" Priority=\"35\" QFormat=\"true\" Name=\"caption\"\/>  <w:LsdException Locked=\"false\" Priority=\"10\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Title\"\/>  <w:LsdException Locked=\"false\" Priority=\"1\" Name=\"Default Paragraph Font\"\/>  <w:LsdException Locked=\"false\" Priority=\"11\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Subtitle\"\/>  <w:LsdException Locked=\"false\" Priority=\"22\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Strong\"\/>  <w:LsdException Locked=\"false\" Priority=\"20\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Emphasis\"\/>  <w:LsdException Locked=\"false\" Priority=\"59\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Table Grid\"\/>  <w:LsdException Locked=\"false\" UnhideWhenUsed=\"false\" Name=\"Placeholder Text\"\/>  <w:LsdException Locked=\"false\" Priority=\"1\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"No Spacing\"\/>  <w:LsdException Locked=\"false\" Priority=\"60\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Shading\"\/>  <w:LsdException Locked=\"false\" Priority=\"61\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light List\"\/>  <w:LsdException Locked=\"false\" Priority=\"62\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Grid\"\/>  <w:LsdException Locked=\"false\" Priority=\"63\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"64\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"65\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"66\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"67\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"68\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"69\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"70\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Dark List\"\/>  <w:LsdException Locked=\"false\" Priority=\"71\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Shading\"\/>  <w:LsdException Locked=\"false\" Priority=\"72\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful List\"\/>  <w:LsdException Locked=\"false\" Priority=\"73\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Grid\"\/>  <w:LsdException Locked=\"false\" Priority=\"60\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Shading Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"61\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light List Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"62\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Grid Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"63\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 1 Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"64\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 2 Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"65\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 1 Accent 1\"\/>  <w:LsdException Locked=\"false\" UnhideWhenUsed=\"false\" Name=\"Revision\"\/>  <w:LsdException Locked=\"false\" Priority=\"34\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"List Paragraph\"\/>  <w:LsdException Locked=\"false\" Priority=\"29\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Quote\"\/>  <w:LsdException Locked=\"false\" Priority=\"30\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Intense Quote\"\/>  <w:LsdException Locked=\"false\" Priority=\"66\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 2 Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"67\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 1 Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"68\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 2 Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"69\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 3 Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"70\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Dark List Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"71\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Shading Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"72\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful List Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"73\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Grid Accent 1\"\/>  <w:LsdException Locked=\"false\" Priority=\"60\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Shading Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"61\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light List Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"62\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Grid Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"63\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 1 Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"64\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 2 Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"65\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 1 Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"66\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 2 Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"67\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 1 Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"68\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 2 Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"69\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 3 Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"70\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Dark List Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"71\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Shading Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"72\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful List Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"73\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Grid Accent 2\"\/>  <w:LsdException Locked=\"false\" Priority=\"60\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Shading Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"61\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light List Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"62\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Grid Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"63\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 1 Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"64\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 2 Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"65\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 1 Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"66\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 2 Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"67\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 1 Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"68\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 2 Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"69\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 3 Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"70\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Dark List Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"71\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Shading Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"72\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful List Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"73\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Grid Accent 3\"\/>  <w:LsdException Locked=\"false\" Priority=\"60\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Shading Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"61\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light List Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"62\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Grid Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"63\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 1 Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"64\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 2 Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"65\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 1 Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"66\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 2 Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"67\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 1 Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"68\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 2 Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"69\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 3 Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"70\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Dark List Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"71\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Shading Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"72\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful List Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"73\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Grid Accent 4\"\/>  <w:LsdException Locked=\"false\" Priority=\"60\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Shading Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"61\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light List Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"62\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Grid Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"63\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 1 Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"64\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 2 Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"65\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 1 Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"66\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 2 Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"67\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 1 Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"68\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 2 Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"69\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 3 Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"70\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Dark List Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"71\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Shading Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"72\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful List Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"73\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Grid Accent 5\"\/>  <w:LsdException Locked=\"false\" Priority=\"60\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Shading Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"61\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light List Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"62\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Light Grid Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"63\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 1 Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"64\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Shading 2 Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"65\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 1 Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"66\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium List 2 Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"67\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 1 Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"68\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 2 Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"69\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Medium Grid 3 Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"70\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Dark List Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"71\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Shading Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"72\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful List Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"73\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" Name=\"Colorful Grid Accent 6\"\/>  <w:LsdException Locked=\"false\" Priority=\"19\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Subtle Emphasis\"\/>  <w:LsdException Locked=\"false\" Priority=\"21\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Intense Emphasis\"\/>  <w:LsdException Locked=\"false\" Priority=\"31\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Subtle Reference\"\/>  <w:LsdException Locked=\"false\" Priority=\"32\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Intense Reference\"\/>  <w:LsdException Locked=\"false\" Priority=\"33\" SemiHidden=\"false\"    UnhideWhenUsed=\"false\" QFormat=\"true\" Name=\"Book Title\"\/>  <w:LsdException Locked=\"false\" Priority=\"37\" Name=\"Bibliography\"\/>  <w:LsdException Locked=\"false\" Priority=\"39\" QFormat=\"true\" Name=\"TOC Heading\"\/> <\/w:LatentStyles><\/xml><![endif]--><!--[if gte mso 10]><![endif]--> <\/p>\n<div style=\"text-align: justify;\"><i>Principal Component Analysis(PCA)<\/i><b>:<\/b> This is a classical method that provides a sequence of best linear approximations to a given high-dimensional observation. It is one of the most popular dimensionality reduction techniques. However, its effectiveness is limited by its global linearity\/<\/div>\n<div style=\"text-align: justify;\"><i>Multidimensional Scaling(MDS)<\/i><b>:<\/b> This technique is closely related to PCA<span style=\"mso-spacerun: yes;\">&nbsp; <\/span>and have the same limitations as PCA.<\/div>\n<div style=\"text-align: justify;\"><i>Factor Analysis<\/i>: This technique assumes that the underlying manifold is a linear subspace. <\/div>\n<div style=\"text-align: justify;\"><i>Independent Component Analysis(ICA)<\/i><b>:<\/b> This technique starts from a factor analysis solution and searches for rotations that lead to independent components.<\/div>\n<div><\/div>\n<div><b>Principal Component Analysis (PCA) <\/b><\/div>\n<div>PCA is a variance-maximising technique that projects the original data&nbsp; onto a direction that maximizes variance. PCA performs a linear mapping of the original data to a lower-dimensional space such that the variance of the data in the low-dimensional representation is maximized.<\/div>\n<div><\/div>\n<div><b>How PCA Works<\/b><\/div>\n<div>In math terms, PCA is the performed by carrying out and eigen-decomposition of the co-variance matrix.<\/div>\n<div>The result would be a set of eigenvectors and a set of eigenvalues which can then be used to describe the original data.&nbsp; <\/div>\n<div><b>A Little More Details<\/b><br \/>An eigenvector in linear algebra is a vector that would not change its direction under associated linear transformation. If we&nbsp; have a non-zero vector v, then its an eigenvector of a square matrix A is Av is a scalar multiple of v.<\/p>\n<p>The eigenvalue is a scalar characteristic value associated with the eigenvector v<br \/>Eigenvectors are the coefficients attached to the eigenvectors and that is what gives the axes their magnitude.<\/p>\n<p><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"8227894917\" style=\"display: block; text-align: center;\"><\/ins> <b>Summary<\/b><br \/>Dimensionality Reduction reduces data in high dimension to lower dimension by obtaining the principal components<\/p>\n<p><i>PCA is performed by:<\/i><\/p>\n<ul>\n<li>constructing a co-variance matrix<\/li>\n<li>performing an eigen-decomposition of that matrix to obtain a set of eigenvectors (W)<\/li>\n<li>columns of W are ordered by the size of their corresponding eigenvalues<\/li>\n<li>choose the first n columns of W and use it to describe your data<\/li>\n<\/ul>\n<p>In the next lesson(which would be a web video), we would actually some of the derivations behind Principal Component Analysis).<br \/>We would also perform PCA on real data using MatLAB and R.<br \/>So you can follow this course to get updates(just click on the follow button under the name of the author) and also subscribe to the video channel <a href=\"https:\/\/www.youtube.com\/channel\/UCvHgEAcw6VpcOA3864pSr5A\">here<\/a>.<\/div>\n<\/div>\n<\/div>\n<p><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"8227894917\" style=\"display: block; text-align: center;\"><\/ins>  <br \/><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"8227894917\" style=\"display: block; text-align: center;\"><\/ins><\/p>\n","protected":false},"excerpt":{"rendered":"<p>More Detailed Video Explanation Here Video on How to Perform PCA in R here We would explain the concept of dimensionality reduction in a very &hellip; <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[11,16,395],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/226"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=226"}],"version-history":[{"count":1,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/226\/revisions"}],"predecessor-version":[{"id":1501,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/226\/revisions\/1501"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=226"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=226"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=226"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}