These are the following\u00a0 eight steps to performing PCA in Python:<\/p>\n
- \n
- Step 1: Import the Neccessary Modules<\/a><\/li>\n
- Step 2: Obtain Your Dataset<\/a><\/li>\n
- Step 3: Preview Your Data<\/a><\/li>\n
- Step 4: Standardize the Data<\/a><\/li>\n
- Step 5: Perform PCA<\/a><\/li>\n
- Step 6: Combine Target and Principal Components<\/a><\/li>\n
- Step 7: Do a Scree Plot of the Principal Components<\/a><\/li>\n
- Step 8: Visualize your New Data in 2D<\/a><\/li>\n<\/ul>\n
<\/p>\n
Step 1: Import the Necessary Modules<\/h5>\n
The modules we would need are pandas, numpy, sklearn and matplotlib. To import them however, write the following import statement inside the first cell of Jupyter Notebook<\/p>\n
\nimport<\/span> pandas<\/span> as<\/span> pd<\/span>\nimport<\/span> numpy<\/span> as<\/span> np<\/span>\nfrom<\/span> sklearn.decomposition<\/span> import<\/span> PCA\nfrom<\/span> sklearn<\/span> import<\/span> preprocessing\nimport<\/span> matplotlib.pyplot<\/span> as<\/span> plt<\/span>\n<\/pre>\n<\/div>\nListing 1.0: Import necessary modules<\/p>\n
<\/p>\n
Step 2: Obtain the Dataset<\/h5>\n
The dataset would be obtained from UCI Machine Learning Repository. To do that, you can right-click on the link below and save a copy of the dataset to your local drive.<\/p>\n
Add the following lines to the next cell to load the dataset into a variable<\/p>\n
<\/p>\n
\nurl =<\/span> \"https:\/\/archive.ics.uci.edu\/ml\/machine-learning-databases\/iris\/iris.data\"<\/span>\n\ndf =<\/span> pd.<\/span>read_csv(url, names=<\/span>['sepal length'<\/span>, 'sepal width'<\/span> , 'petal lenght'<\/span>, 'petal width'<\/span>, 'target'<\/span>])\n<\/pre>\n<\/div>\nListing 1.1: Obtain and load your dataset<\/p>\n
In Listing 1.1, the first line specifies the url of the dataset, the second line loads the dataset into a dataframe df (a dataframe is simply used to hold data).<\/p>\n
pd.read_csv()<\/strong> is a function in pandas. The first argument is the path to the data, the second argument is a list of the column names. What this means is the that the first column of the data would be named ‘sepal lenght’, the second column is ‘sepal_width’ and so on.<\/p>\n
When the code in Listing 2.1 executes, then your dataset is available in the variable df.<\/p>\n
<\/p>\n
Step 3: Preview Your Data<\/h5>\n
So you can view your data by typing df into the next cell and running it as shown in Figure 1.0. You can also type print(df). In the table, there are four features, and one target(or class)<\/p>\n
![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/DataFrame-in-Python.jpg\")
Figure 1.0: Data in DataFrame<\/figcaption><\/figure>\n <\/p>\n
Step 4: Perform Scaling on the Data<\/h5>\n
This means that we need to center and scale the data. In this way the average value of each record would be 0 and the variance for each record would be 1.<\/p>\n
To scale our data, we would use StandardScalar which is available in sklearn.<\/p>\n
Note that we are only going to scale the features and not the target.<\/span> So to do this, we<\/p>\n
- \n
- first import StandardScalar<\/li>\n
- separate the features from the target<\/li>\n
- scale the features<\/li>\n<\/ul>\n
This three operations are accomplished using the four lines of codes below<\/p>\n
<\/p>\n
\nfrom<\/span> sklearn.preprocessing<\/span> import<\/span> StandardScaler\n\nfeatures =<\/span> ['sepal length'<\/span>, 'sepal width'<\/span>, 'petal length'<\/span>, 'petal width'<\/span>]\n\nx =<\/span> df.<\/span>loc[:, features].<\/span>values\n\ny =<\/span> df.<\/span>loc[:, ['target'<\/span>]].<\/span>values\n\nx =<\/span> StandardScaler().<\/span>fit_transform(x)\n<\/pre>\n<\/div>\nListing 1.2: Separate features from target and standardize features<\/p>\n
<\/p>\n
Step 5:\u00a0 Perform PCA<\/h5>\n
To then perform PCA we would use PCA module from sklearn which we have already imported in Step 1. In Listing 1.3, below, the first and the\u00a0 line performs the PCA, the third line loads the principal components into a dataframe. You can view your data by typing principalComponents or principalDataframe in a cell and running it.<\/p>\n
<\/p>\n
\npca =<\/span> PCA(n_components=<\/span>2<\/span>)\n\nprincipalComponents =<\/span> pca.<\/span>fit_transform(x)\n\nprincipalDataframe =<\/span> pd.<\/span>DataFrame(data =<\/span> principalComponents, columns =<\/span> ['PC1'<\/span>, 'PC2'<\/span>])\n<\/pre>\n<\/div>\nListing 1.3: PCA for two Principal Components<\/p>\n
<\/p>\n
Step 6: Combine the Target and the Principal Components<\/h5>\n
Remember that the original data has five columns: four features and one target column. Now after performing PCA, we have just two columns for the features. The target dataset y was not touched. Therefore, we attached back the target column to the new set of principal components. To do that, use the code below.<\/span><\/p>\n
\ntargetDataframe =<\/span> df[['target'<\/span>]]\n\nnewDataframe =<\/span> pd.<\/span>concat([principalDataframe, targetDataframe],axis =<\/span> 1<\/span>)\n<\/pre>\n<\/div>\nListing 1.4: Combine Principal Components with target<\/p>\n
You can also view your new dataset by just typing newDataframe and running the cell.\u00a0 Your output would therefore be as shown in Figure 1.1<\/p>\n
![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/newDataframe.jpg\")
Figure 1.1: New Dataset after performing PCA<\/figcaption><\/figure>\n <\/p>\n
Step 7: Perform a Scree Plot of the Principal Components<\/h5>\n
A scree plot is like a bar chart showing the size of each of the principal components. It helps us to visualize the percentage of variation captured by each of the principal components. To perform a scree plot you need to:<\/p>\n
- \n
- first of all, create a list of columns<\/li>\n
- \u00a0then, list of PCs<\/li>\n
- finally, do the scree plot using plt<\/li>\n<\/ul>\n
Now, copy and past the code in listing 1.5 below into Jupyter Notebook and then run it. Consequently, your output would be as shown in Figure 1.2<\/p>\n
<\/p>\n
\npercent_variance =<\/span> np.<\/span>round(pca.<\/span>explained_variance_ratio_*<\/span> 100<\/span>, decimals =<\/span>2<\/span>)\ncolumns =<\/span> ['PC1'<\/span>, 'PC2'<\/span>, 'PC3'<\/span>, 'PC4'<\/span>]\nplt.<\/span>bar(x=<\/span> range<\/span>(1<\/span>,5<\/span>), height=<\/span>percent_variance, tick_label=<\/span>columns)\nplt.<\/span>ylabel('Percentate of Variance Explained'<\/span>)\nplt.<\/span>xlabel('Principal Component'<\/span>)\nplt.<\/span>title('PCA Scree Plot'<\/span>)\nplt.<\/span>show()\n<\/pre>\n<\/div>\nListing 1.5: PCA Scree Plot<\/p>\n
You can hence see the scree plot below.<\/p>\n
![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/Scree-Plot-1024x483.jpg\")
Figure 1.3: Scree Plot<\/figcaption><\/figure>\n <\/p>\n
Step 8: Plot the Principal Components on 2D<\/h5>\n
Now we have performed PCA, we need to visualize the new dataset to see how PCA makes it easier to explain the original data. We would use scatter plot<\/p>\n
<\/p>\n
\nplt.<\/span>scatter(principalDataframe.<\/span>PC1, principalDataframe.<\/span>PC2)\nplt.<\/span>title('PC1 against PC2'<\/span>)\nplt.<\/span>xlabel('PC1'<\/span>)\nplt.<\/span>ylabel('PC2'<\/span>)\n<\/pre>\n<\/div>\nListing 1.6:\u00a0 2D Plot of PC1 and PC2<\/p>\n
If you execute the code above then you will have the plot given in Figure 1.2<\/p>\n
![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/PCA-Plot-1.jpg\")
Figure 1.3: First PCA plot of PC1 and PC2<\/figcaption><\/figure>\n So what have we achieved?<\/strong><\/p>\n
We would repeat this plot this time with colors for each of the targets (Iris-setosa, Iris-versicolor and Iris-virginica). In this way we would see how PCA helps explain the data. However, to keep things simple, I would not explain this very code.\u00a0 Write and run the code below.<\/p>\n
<\/p>\n
\nfig =<\/span> plt.<\/span>figure(figsize =<\/span> (8<\/span>,8<\/span>))\nax =<\/span> fig.<\/span>add_subplot(1<\/span>,1<\/span>,1<\/span>) \nax.<\/span>set_xlabel('PC1'<\/span>)\nax.<\/span>set_ylabel('PC2'<\/span>)\n\nax.<\/span>set_title('Plot of PC1 vs PC2'<\/span>, fontsize =<\/span> 20<\/span>)\n\ntargets =<\/span> ['Iris-setosa'<\/span>, 'Iris-versicolor'<\/span>, 'Iris-virginica'<\/span>]\n\ncolors =<\/span> ['r'<\/span>, 'g'<\/span>, 'b'<\/span>]\n\nfor<\/span> target, color in<\/span> zip<\/span>(targets,colors):\n indicesToKeep =<\/span> newDataframe['target'<\/span>] ==<\/span> target\n ax.<\/span>scatter(newDataframe.<\/span>loc[indicesToKeep, 'PC1'<\/span>]\n , newDataframe.<\/span>loc[indicesToKeep, 'PC2'<\/span>]\n , c =<\/span> color\n , s =<\/span> 50<\/span>)\n \nax.<\/span>legend(targets)\nax.<\/span>grid()\n<\/pre>\n<\/div>\nListing 1.7: Plot of PC1 vs PC2 with<\/span> color codes<\/p>\n
Likewise, if you execute the code in Listing 1.6 above, you will have the output given in Figure below:<\/p>\n
![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/PCA-Plot-2-1024x682.jpg\")
Figure 1.4: Final Plot of PC1 and PC2<\/figcaption><\/figure>\n Explaining the Variance Using Principal Component<\/strong><\/p>\n
Finally\u00a0 we need to see how the two principal components explain our data. To do that we would use the command below:<\/p>\n
<\/p>\n
\npca.<\/span>explained_variance_ratio_\n<\/pre>\n<\/div>\n<\/p>\n
Then you will get the output:<\/p>\n
\n\n
- Step 2: Obtain Your Dataset<\/a><\/li>\n