{"id":276,"date":"2018-12-31T11:05:55","date_gmt":"2018-12-31T11:05:55","guid":{"rendered":"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/?post_type=chapter&#038;p=276"},"modified":"2018-12-31T11:41:18","modified_gmt":"2018-12-31T11:41:18","slug":"principal-component-analysis-using-r","status":"publish","type":"chapter","link":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/chapter\/principal-component-analysis-using-r\/","title":{"rendered":"Principal Component Analysis using R"},"content":{"raw":"<div><span style=\"float: right\"><a href=\"https:\/\/youtu.be\/W6oaWajCIG8\" target=\"_blank\" rel=\"noopener\"><img src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"epgp books\" width=\"75px\" height=\"75px;\" \/><\/a>\r\n<\/span><\/div>\r\n<div>\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n<strong>1\u00a0 Introduction<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">A principal component analysis is concerned with explaining the variance-covariance structure of a set of variables through a fewer linear combinations of these vari-ables.<\/p>\r\n&nbsp;\r\n\r\nIts objectives are:\r\n<ul>\r\n \t<li>Data reduction<\/li>\r\n \t<li>Interpretation<\/li>\r\n<\/ul>\r\n<\/div>\r\n<strong><span style=\"text-align: initial;font-size: 1em\">2 Quick Review<\/span><\/strong>\r\n<div><img class=\"alignleft size-full wp-image-281\" style=\"text-align: initial;font-size: 1em\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-1.png\" alt=\"\" width=\"575\" height=\"586\" \/><\/div>\r\n<div>\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n<strong><span style=\"text-align: initial;font-size: 1em\">A 2-Dimensional Example<\/span><\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">First<span style=\"text-align: initial;font-size: 1em\"> we are going to consider a <\/span>two dimensional<span style=\"text-align: initial;font-size: 1em\"> example and thereby look at the geometric interpretation of Principal Component Analysis<\/span><\/p>\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">We get some data in R arbitrarily like say<\/span>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">x &lt;- c(2.5,0.5,2.2,1.9,3.1,2.3,2,1,1.5,1.1)<\/span>\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">y &lt;- c(2.4,0.7,2.9,2.2,3.0,2.7,1.6,1.1,1.6,0.9)<\/span>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">Let us see how the plot of the data looks like<\/span>\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">We use the R command plot() to plot the data<\/span>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">plot(x,y,xlim=c(-1,4),ylim=c(-1,4))<\/span>\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">abline(h=0,v=0,lty=3)<\/span>\r\n\r\n<\/div>\r\n<img class=\"size-full wp-image-283 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-2.png\" alt=\"\" width=\"456\" height=\"515\" \/>\r\n<div>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">For PCA to work properly, we must subtract the mean for each dimension This produces a data set whose mean is zero<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We subtract the mean of x values from the components of x and also the same mean centering is done for <\/span>the y<span style=\"text-align: initial;font-size: 1em\"> values<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">x1 &lt;- x - mean(x)<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">y1 &lt;- y - mean(y)<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We plot this mean centered data<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">plot(x1,y1,main=\"Plot of the centered data\")<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(h=0,v=0,lty=3)<\/span><\/p>\r\n\r\n<\/div>\r\n<img class=\"size-full wp-image-284 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-3.png\" alt=\"\" width=\"499\" height=\"507\" \/>\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">Before proceeding further we <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> take a look at what is meant by an orthogonal transformation<\/span>\r\n\r\n&nbsp;\r\n\r\n<strong><span style=\"text-align: initial;font-size: 1em\">What is an Orthogonal Transformation?<\/span><\/strong>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">Consider a linear transformation from the variables (x,y)!(u,v) where,<\/span>\r\n<div>\r\n\r\n<img class=\"alignleft size-full wp-image-285\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-4.png\" alt=\"\" width=\"541\" height=\"138\" \/>\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n<strong>What effect does an orthogonal transformation have on the axes??<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">Consider the mean centered data as obtained previously. We consider an or-thogonal transformation of the data and we see how the plot of the new data looks like<\/p>\r\n\r\n<\/div>\r\n<img class=\"size-full wp-image-286 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-5.png\" alt=\"\" width=\"410\" height=\"440\" \/>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">The new data when plotted looks like the above with the red and green <\/span>coloured<span style=\"text-align: initial;font-size: 1em\"> axes as the reference axes. That is the (u,v) axes are shown as the red and green <\/span>coloured<span style=\"text-align: initial;font-size: 1em\"> dotted lines respectively.<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Also in the same <\/span>plot<span style=\"text-align: initial;font-size: 1em\"> we have the original set of axes, as denoted by the black dotted lines, i.e the (x,y) axes<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Note that the linear transformation of the data has basically led to a <\/span>rotation<span style=\"text-align: initial;font-size: 1em\"> of the axes. And here the orthogonal transformation, apart from bringing about a rotation of the axes have also maintained the orthogonality of the axes<\/span><\/p>\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">If we further rotate the axes we would have the following\u00a0 fi<\/span>gure\r\n<div>\r\n\r\n<img class=\"size-full wp-image-287 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-6.png\" alt=\"\" width=\"412\" height=\"441\" \/>\r\n\r\n<strong><span style=\"text-align: initial;font-size: 1em\">Back to PCA<\/span><\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">We nd<span style=\"text-align: initial;font-size: 1em\"> the <\/span>eigen values<span style=\"text-align: initial;font-size: 1em\"> and the corresponding normalized <\/span>eigen vectors<span style=\"text-align: initial;font-size: 1em\"> of the var-<\/span>cov<span style=\"text-align: initial;font-size: 1em\"> matrix of the mean centered data (var-<\/span>cov<span style=\"text-align: initial;font-size: 1em\"> matrix of mean centered data is the same as the correlation matrix of the original data)<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">m &lt;- as.matrix(cbind(x1,y1))<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">cov.m &lt;- cov(m)<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">cov.eig &lt;- eigen(cov.m)<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">m.values &lt;- cov.eig$values\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 #We get the eigen values<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">#Cumulative percentage of variability explained by the principal components\u00a0<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">(cumsum(m.values)\/sum(m.values))*100<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">[1]\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 96.31813 100.00000<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Thus we see that 96% of <\/span>total<span style=\"text-align: initial;font-size: 1em\"> variability is explained by the <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> principal component<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We plot the mean centered data along with the principal components<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Note that the <\/span>eigen vectors<span style=\"text-align: initial;font-size: 1em\"> give the direction of the principal components<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">So here we plot the direction of principal components as lines through the origin and the slope of the line is proportional to the slope of the <\/span>eigen vectors<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">plot(x1,y1)<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(h=0,v=0,lty=3)<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(a=0,b=(cov.eig$vectors[1,1]\/cov.eig$vectors[2,1]), col=\"red\")<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(a=0,b=(cov.eig$vectors[1,2]\/cov.eig$vectors[2,2]), col=\"green\")<\/span><\/p>\r\n\r\n<\/div>\r\n<img class=\"size-full wp-image-288 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-7.png\" alt=\"\" width=\"439\" height=\"423\" \/>\r\n<div>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">The <\/span>eigen vector<span style=\"text-align: initial;font-size: 1em\"> corresponding to the largest <\/span>eigen value<span style=\"text-align: initial;font-size: 1em\"> is obtained as<\/span>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">p1 &lt;- cov.eig$vectors[,1]\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 # Eigen vector corr to largest eigen value\u00a0<\/span><span style=\"text-align: initial;font-size: 1em\">p1<\/span>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">[1] 0.6778734 0.7351787<\/span>\r\n\r\n&nbsp;\r\n\r\n<strong><span style=\"text-align: initial;font-size: 1em\">The First Principal Component is given by<\/span><\/strong>\r\n\r\n&nbsp;\r\n\r\n<span style=\"text-align: initial;font-size: 1em\">Y1==0.6778734*x + 0.7351787*y<\/span>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We can optionally recover the original data back, by 100 % if we choose all components, or an approximation otherwise<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Let us consider that we have the data on only the First Principal Component To get back to the original dataset, we actually invert the orthogonal <\/span>transfor-mation<span style=\"text-align: initial;font-size: 1em\">. The R code is as follows<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Y1 &lt;- as.numeric(p1 %*% t(m)) # new dataset for feature vector 1 <\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1 &lt;- t(p1 %*% t(Y1))<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1[,1] &lt;- original.dataset1[,1] + mean(x) # re-add means <\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1[,2] &lt;- original.dataset1[,2] + mean(y)<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1<\/span><\/p>\r\n\r\n<\/div>\r\n<div>\r\n\r\n<img class=\"alignleft size-full wp-image-289\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-8.png\" alt=\"\" width=\"150\" height=\"213\" \/>\r\n\r\n<\/div>\r\n<div><\/div>\r\n&nbsp;\r\n<div>\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\nLet us plot the data obtained\r\n\r\n&nbsp;\r\n\r\nplot(original.dataset1[,1],original.dataset1[,2],xlim=c(-1,4),ylim=c(-1,4),\r\n\r\ntype=\"p\",main=\"Plot of Data\",xlab=\"X\",ylab=\"Y\")\r\n\r\nabline(h=0,v=0,lty=3)\r\n\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-290 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-9.png\" alt=\"\" width=\"408\" height=\"475\" \/>\r\n<p style=\"text-align: justify\">Notice that in the approximation the variation over the 2nd eigenvector is gone as expected (since it was previously erased)<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong>Example<span style=\"text-align: initial;font-size: 1em\"> where PCA is worthwhile<\/span><\/strong><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Consider the data on the times taken by some athletes to complete di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> category races like 100m race, 200m race etc<\/span><\/p>\r\n\r\n<\/div>\r\n<div>\r\n\r\n<img class=\"alignleft size-full wp-image-291\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-10.png\" alt=\"\" width=\"544\" height=\"372\" \/>\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n<\/div>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Notice that the time in the <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> 3 categories of races ( <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> three columns ) are given in seconds whereas the other times are given in minutes<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">There are several functions from di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> packages for performing PCA :<\/span><\/p>\r\n\r\n<ul>\r\n \t<li style=\"text-align: justify\">prcomp<span style=\"text-align: initial;font-size: 1em\">() and <\/span>princomp<span style=\"text-align: initial;font-size: 1em\">() from the built-in R stats package <\/span><\/li>\r\n \t<li style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">PCA() from FactoMineR package<\/span><\/li>\r\n \t<li style=\"text-align: justify\">dudi.pca() from ade4 package<\/li>\r\n<\/ul>\r\n<div>\r\n\r\n\u00a0 \u00a0\u00a0princomp() uses the spectral decomposition approach\r\n\r\nThe functionsprcomp()andPCA() [FactoMineR] use the singular value decom-position (SVD)\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">According to R help, SVD has slightly better numerical accuracy. Therefore, prcomp() is the preferred function<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Let us read the data into R and we use the R function <\/span>prcomp<span style=\"text-align: initial;font-size: 1em\">()<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">runner.pca=prcomp(data,scale=TRUE)<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">names(runner.pca)<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">[1] \"sdev\"\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"rotation\" \"center\"\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"scale\"\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"x\"<\/span><\/p>\r\n\r\n<\/div>\r\n<div>\r\n\r\n&nbsp;\r\n\r\n&gt; runner.pca$sdev\r\n\r\n&nbsp;\r\n\r\n[1]\u00a0\u00a0\u00a0\u00a0 2.5733531 0.9368128 0.3991505 0.3522065 0.2826310 0.2607013 0.2154519\r\n\r\n[8]\u00a0\u00a0\u00a0\u00a0 0.1503333\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The argument scale=TRUE speci es that the observations are rst standardized and then PCA is performed. This standardization is required since the variables are in di erent units.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">names(runner.pca)<\/p>\r\n<p style=\"text-align: justify\">[1] \"sdev\"\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"rotation\" \"center\"\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"scale\"\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"x\"<\/p>\r\n&nbsp;\r\n\r\nrunner.pca$sdev\r\n\r\n&nbsp;\r\n\r\n[1]\u00a0\u00a0\u00a0\u00a0 2.5733531 0.9368128 0.3991505 0.3522065 0.2826310 0.2607013 0.2154519\r\n\r\n[8]\u00a0\u00a0\u00a0\u00a0 0.1503333\r\n\r\n&nbsp;\r\n\r\nWhen we extract the output sdev from the R object runner.pca we get the square roots of the eigen values of the correlation matrix corresponding to the data\r\n\r\n&nbsp;\r\n\r\nrunner.pca$rotation\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The columns given by extracting the output rotation from the R object run-ner.pca are the eigen vectors corresponding to the eigen values<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">So to obtain the <\/span>eigen values<span style=\"text-align: initial;font-size: 1em\"> we square the '<\/span>sdev<span style=\"text-align: initial;font-size: 1em\">'<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We also obtain the percentage of variability explained by the di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> principal components and the cumulative percentage of variability explained from the formula only.<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\"># Eigenvalues\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0eig &lt;- (runner.pca$sdev)^2<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\"># Variances in percentage\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 variance &lt;- eig*100\/sum(eig)<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\"># Cumulative variances\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 cumvar &lt;- cumsum(variance)<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">runner.eig&lt;- data.frame(eig = eig, variance = variance, cumvariance = cumvar) runner.eig<\/span><\/p>\r\n\r\n<\/div>\r\n<div>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td>eig<\/td>\r\n<td>variance<\/td>\r\n<td>cumvariance<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>6.62214613<\/td>\r\n<td>82.7768266<\/td>\r\n<td>82.77683<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>0.87761829<\/td>\r\n<td>10.9702287<\/td>\r\n<td>93.74706<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>0.15932114<\/td>\r\n<td>1.9915143<\/td>\r\n<td>95.73857<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>0.12404939<\/td>\r\n<td>1.5506173<\/td>\r\n<td>97.28919<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>0.07988027<\/td>\r\n<td>0.9985034<\/td>\r\n<td>98.28769<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>0.06796515<\/td>\r\n<td>0.8495644<\/td>\r\n<td>99.13725<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>0.04641953<\/td>\r\n<td>0.5802441<\/td>\r\n<td>99.71750<\/td>\r\n<\/tr>\r\n<tr>\r\n<td>0.02260010<\/td>\r\n<td>0.2825012<\/td>\r\n<td>100.00000<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n<div><strong><span style=\"font-size: 1em;text-align: initial\">\u00a0 \u00a0 Scree Plot Using Base Graphics<\/span><\/strong><\/div>\r\n<div><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 To draw the Scree plot we can use the base graphics functions like barplot(), lines() <\/span>etc.<\/div>\r\n<div><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 The R code is as follows<\/span><\/div>\r\n<div><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 barplot(runner.eig[, 2], names.arg=1:nrow(runner.eig), main = \"Variances\",<\/span><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 xlab = \"Principal Components\",<\/span><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 ylab = \"Percentage of variances\",<\/span><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 col =\"steelblue\")<\/span><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 # Add connected line segments to the plot<\/span><\/div>\r\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 lines(x = 1:nrow(runner.eig), runner.eig[, 2], type=\"b\", pch=19, col = \"red\")<\/span><\/div>\r\n<div><\/div>\r\n<div><img class=\"size-full wp-image-292 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11.png\" alt=\"\" width=\"481\" height=\"480\" \/><\/div>\r\n<div>\r\n\r\n\u00a0 \u00a0 <strong>Note that:<\/strong>\r\n<ul>\r\n \t<li>The 2 principal components retain 94% of original data variability<\/li>\r\n \t<li>Instead of 8 original variables<span style=\"text-align: initial;font-size: 1em\"> we may work with 2 principal components<\/span><\/li>\r\n<\/ul>\r\n<strong>We can achieve <\/strong>signi<strong> cant Reduction in Dimension by PCA<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">There are several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries which aid in PCA. The package <\/span>factoextra<span style=\"text-align: initial;font-size: 1em\"> is used for the visualization of <\/span>theprincipal<span style=\"text-align: initial;font-size: 1em\"> component analysis results.<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">factoextra can be installed and loaded as follow :<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">install.packages(\"<\/span>devtools<span style=\"text-align: initial;font-size: 1em\">\")<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">devtools::install_github(\"kassambara\/factoextra\")<\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">#\u00a0\u00a0 load <\/span><\/p>\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">library(\"factoextra\")<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong style=\"text-align: initial;font-size: 1em\">Scree plot using Factoextra:<\/strong><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">The R code is <\/span>fviz screeplot<span style=\"text-align: initial;font-size: 1em\">()<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">fviz_screeplot(runner.pca, ncp=10)<\/span><\/p>\r\n\r\n<\/div>\r\n<img class=\"size-full wp-image-293 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-12.png\" alt=\"\" width=\"524\" height=\"455\" \/>\r\n<div>\r\n\r\n&nbsp;\r\n\r\nExample<strong><span style=\"text-align: initial;font-size: 1em\"> where PCA is not worthwhile<\/span><\/strong>\r\n\r\n<\/div>\r\n<div>\r\n\r\n<img class=\"alignleft size-full wp-image-294\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-13.png\" alt=\"\" width=\"649\" height=\"519\" \/>\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\nNote that:\r\n\r\n&nbsp;\r\n\r\nThe\u00a0 first 5 principal components together explain 93% of the total variability\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">So we do not achieve any significant reduction in dimension by PCA for this problem. This may be attributed to the fact that the variables may have very low correlations amongst themselves.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Scree Plot<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">par(mar=c(5.1,4.85,4.1,2.1))<\/span><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">plot(1:6,eigen.values, type=\"o\", main=\"Scree Plot\", xlab=\"Index\", ylab=expression(lambda[j]), cex.lab=1.5, col=\"dark red\",lwd=2)<\/span><\/p>\r\n\r\n<\/div>\r\n<div>\r\n\r\n<img class=\"size-full wp-image-295 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-14.png\" alt=\"\" width=\"536\" height=\"474\" \/>\r\n<p style=\"text-align: justify\">Even from the Scree plot we see that the rst bend in the curve occurs for index 2, however at such a point the jump is not signi cantly close to 0. This bend and signi cant jump can be seen almost at index 5<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong><span style=\"text-align: initial;font-size: 1em\">SUMMARY<\/span><\/strong><\/p>\r\n\r\n<ul>\r\n \t<li style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">In R we can perform Principal Component Analysis from the <\/span>rst princi-ples<span style=\"text-align: initial;font-size: 1em\"> or using several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries<\/span><\/li>\r\n \t<li style=\"text-align: justify\">In R we can perform Principal Component Analysis from the rst princi-ples<span style=\"text-align: initial;font-size: 1em\"> or using several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries<\/span><\/li>\r\n \t<li style=\"text-align: justify\">In R we can perform Principal Component Analysis from the rst princi-ples<span style=\"text-align: initial;font-size: 1em\"> or using several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries<\/span><\/li>\r\n<\/ul>\r\n<\/div>\r\n&nbsp;\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td><strong>you can view video on Principal Component Analysis using R<\/strong><\/td>\r\n<td><a href=\"https:\/\/youtu.be\/W6oaWajCIG8\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-120\" src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"\" width=\"36\" height=\"36\" \/><\/a><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n&nbsp;\r\n\r\n<strong>References<\/strong>\r\n<ul>\r\n \t<li>R.A.Johnson &amp; D.W. Wichern, Applied Multivariate Statistical Analysis, Pearson<\/li>\r\n \t<li>T.W. Anderson, An Introduction to Multivariate Analysis, John Wiley<\/li>\r\n \t<li>G.A.F. Seber, Multivariate Observations, John Wiley<\/li>\r\n \t<li>N.C. Giri, Multivariate Statistical Inference, Academic Press<\/li>\r\n<\/ul>","rendered":"<div><span style=\"float: right\"><a href=\"https:\/\/youtu.be\/W6oaWajCIG8\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"epgp books\" width=\"75px\" height=\"75px;\" \/><\/a><br \/>\n<\/span><\/div>\n<div>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><strong>1\u00a0 Introduction<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">A principal component analysis is concerned with explaining the variance-covariance structure of a set of variables through a fewer linear combinations of these vari-ables.<\/p>\n<p>&nbsp;<\/p>\n<p>Its objectives are:<\/p>\n<ul>\n<li>Data reduction<\/li>\n<li>Interpretation<\/li>\n<\/ul>\n<\/div>\n<p><strong><span style=\"text-align: initial;font-size: 1em\">2 Quick Review<\/span><\/strong><\/p>\n<div><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-281\" style=\"text-align: initial;font-size: 1em\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-1.png\" alt=\"\" width=\"575\" height=\"586\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-1.png 575w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-1-294x300.png 294w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-1-65x66.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-1-225x229.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-1-350x357.png 350w\" sizes=\"auto, (max-width: 575px) 100vw, 575px\" \/><\/div>\n<div>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><strong><span style=\"text-align: initial;font-size: 1em\">A 2-Dimensional Example<\/span><\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">First<span style=\"text-align: initial;font-size: 1em\"> we are going to consider a <\/span>two dimensional<span style=\"text-align: initial;font-size: 1em\"> example and thereby look at the geometric interpretation of Principal Component Analysis<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">We get some data in R arbitrarily like say<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">x &lt;- c(2.5,0.5,2.2,1.9,3.1,2.3,2,1,1.5,1.1)<\/span><\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">y &lt;- c(2.4,0.7,2.9,2.2,3.0,2.7,1.6,1.1,1.6,0.9)<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">Let us see how the plot of the data looks like<\/span><\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">We use the R command plot() to plot the data<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">plot(x,y,xlim=c(-1,4),ylim=c(-1,4))<\/span><\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">abline(h=0,v=0,lty=3)<\/span><\/p>\n<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-283 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-2.png\" alt=\"\" width=\"456\" height=\"515\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-2.png 456w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-2-266x300.png 266w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-2-65x73.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-2-225x254.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-2-350x395.png 350w\" sizes=\"auto, (max-width: 456px) 100vw, 456px\" \/><\/p>\n<div>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">For PCA to work properly, we must subtract the mean for each dimension This produces a data set whose mean is zero<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We subtract the mean of x values from the components of x and also the same mean centering is done for <\/span>the y<span style=\"text-align: initial;font-size: 1em\"> values<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">x1 &lt;- x &#8211; mean(x)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">y1 &lt;- y &#8211; mean(y)<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We plot this mean centered data<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">plot(x1,y1,main=&#8221;Plot of the centered data&#8221;)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(h=0,v=0,lty=3)<\/span><\/p>\n<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-284 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-3.png\" alt=\"\" width=\"499\" height=\"507\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-3.png 499w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-3-295x300.png 295w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-3-65x66.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-3-225x229.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-3-350x356.png 350w\" sizes=\"auto, (max-width: 499px) 100vw, 499px\" \/><\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">Before proceeding further we <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> take a look at what is meant by an orthogonal transformation<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><strong><span style=\"text-align: initial;font-size: 1em\">What is an Orthogonal Transformation?<\/span><\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">Consider a linear transformation from the variables (x,y)!(u,v) where,<\/span><\/p>\n<div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-285\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-4.png\" alt=\"\" width=\"541\" height=\"138\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-4.png 541w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-4-300x77.png 300w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-4-65x17.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-4-225x57.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-4-350x89.png 350w\" sizes=\"auto, (max-width: 541px) 100vw, 541px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><strong>What effect does an orthogonal transformation have on the axes??<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">Consider the mean centered data as obtained previously. We consider an or-thogonal transformation of the data and we see how the plot of the new data looks like<\/p>\n<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-286 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-5.png\" alt=\"\" width=\"410\" height=\"440\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-5.png 410w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-5-280x300.png 280w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-5-65x70.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-5-225x241.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-5-350x376.png 350w\" sizes=\"auto, (max-width: 410px) 100vw, 410px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">The new data when plotted looks like the above with the red and green <\/span>coloured<span style=\"text-align: initial;font-size: 1em\"> axes as the reference axes. That is the (u,v) axes are shown as the red and green <\/span>coloured<span style=\"text-align: initial;font-size: 1em\"> dotted lines respectively.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Also in the same <\/span>plot<span style=\"text-align: initial;font-size: 1em\"> we have the original set of axes, as denoted by the black dotted lines, i.e the (x,y) axes<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Note that the linear transformation of the data has basically led to a <\/span>rotation<span style=\"text-align: initial;font-size: 1em\"> of the axes. And here the orthogonal transformation, apart from bringing about a rotation of the axes have also maintained the orthogonality of the axes<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">If we further rotate the axes we would have the following\u00a0 fi<\/span>gure<\/p>\n<div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-287 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-6.png\" alt=\"\" width=\"412\" height=\"441\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-6.png 412w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-6-280x300.png 280w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-6-65x70.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-6-225x241.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-6-350x375.png 350w\" sizes=\"auto, (max-width: 412px) 100vw, 412px\" \/><\/p>\n<p><strong><span style=\"text-align: initial;font-size: 1em\">Back to PCA<\/span><\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">We nd<span style=\"text-align: initial;font-size: 1em\"> the <\/span>eigen values<span style=\"text-align: initial;font-size: 1em\"> and the corresponding normalized <\/span>eigen vectors<span style=\"text-align: initial;font-size: 1em\"> of the var-<\/span>cov<span style=\"text-align: initial;font-size: 1em\"> matrix of the mean centered data (var-<\/span>cov<span style=\"text-align: initial;font-size: 1em\"> matrix of mean centered data is the same as the correlation matrix of the original data)<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">m &lt;- as.matrix(cbind(x1,y1))<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">cov.m &lt;- cov(m)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">cov.eig &lt;- eigen(cov.m)<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">m.values &lt;- cov.eig$values\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 #We get the eigen values<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">#Cumulative percentage of variability explained by the principal components\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">(cumsum(m.values)\/sum(m.values))*100<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">[1]\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 96.31813 100.00000<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Thus we see that 96% of <\/span>total<span style=\"text-align: initial;font-size: 1em\"> variability is explained by the <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> principal component<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We plot the mean centered data along with the principal components<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Note that the <\/span>eigen vectors<span style=\"text-align: initial;font-size: 1em\"> give the direction of the principal components<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">So here we plot the direction of principal components as lines through the origin and the slope of the line is proportional to the slope of the <\/span>eigen vectors<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">plot(x1,y1)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(h=0,v=0,lty=3)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(a=0,b=(cov.eig$vectors[1,1]\/cov.eig$vectors[2,1]), col=&#8221;red&#8221;)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">abline(a=0,b=(cov.eig$vectors[1,2]\/cov.eig$vectors[2,2]), col=&#8221;green&#8221;)<\/span><\/p>\n<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-288 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-7.png\" alt=\"\" width=\"439\" height=\"423\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-7.png 439w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-7-300x289.png 300w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-7-65x63.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-7-225x217.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-7-350x337.png 350w\" sizes=\"auto, (max-width: 439px) 100vw, 439px\" \/><\/p>\n<div>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">The <\/span>eigen vector<span style=\"text-align: initial;font-size: 1em\"> corresponding to the largest <\/span>eigen value<span style=\"text-align: initial;font-size: 1em\"> is obtained as<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">p1 &lt;- cov.eig$vectors[,1]\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 # Eigen vector corr to largest eigen value\u00a0<\/span><span style=\"text-align: initial;font-size: 1em\">p1<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">[1] 0.6778734 0.7351787<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><strong><span style=\"text-align: initial;font-size: 1em\">The First Principal Component is given by<\/span><\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-align: initial;font-size: 1em\">Y1==0.6778734*x + 0.7351787*y<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We can optionally recover the original data back, by 100 % if we choose all components, or an approximation otherwise<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Let us consider that we have the data on only the First Principal Component To get back to the original dataset, we actually invert the orthogonal <\/span>transfor-mation<span style=\"text-align: initial;font-size: 1em\">. The R code is as follows<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Y1 &lt;- as.numeric(p1 %*% t(m)) # new dataset for feature vector 1 <\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1 &lt;- t(p1 %*% t(Y1))<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1[,1] &lt;- original.dataset1[,1] + mean(x) # re-add means <\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1[,2] &lt;- original.dataset1[,2] + mean(y)<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">original.dataset1<\/span><\/p>\n<\/div>\n<div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-289\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-8.png\" alt=\"\" width=\"150\" height=\"213\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-8.png 150w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-8-65x92.png 65w\" sizes=\"auto, (max-width: 150px) 100vw, 150px\" \/><\/p>\n<\/div>\n<div><\/div>\n<p>&nbsp;<\/p>\n<div>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>Let us plot the data obtained<\/p>\n<p>&nbsp;<\/p>\n<p>plot(original.dataset1[,1],original.dataset1[,2],xlim=c(-1,4),ylim=c(-1,4),<\/p>\n<p>type=&#8221;p&#8221;,main=&#8221;Plot of Data&#8221;,xlab=&#8221;X&#8221;,ylab=&#8221;Y&#8221;)<\/p>\n<p>abline(h=0,v=0,lty=3)<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-290 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-9.png\" alt=\"\" width=\"408\" height=\"475\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-9.png 408w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-9-258x300.png 258w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-9-65x76.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-9-225x262.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-9-350x407.png 350w\" sizes=\"auto, (max-width: 408px) 100vw, 408px\" \/><\/p>\n<p style=\"text-align: justify\">Notice that in the approximation the variation over the 2nd eigenvector is gone as expected (since it was previously erased)<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong>Example<span style=\"text-align: initial;font-size: 1em\"> where PCA is worthwhile<\/span><\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Consider the data on the times taken by some athletes to complete di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> category races like 100m race, 200m race etc<\/span><\/p>\n<\/div>\n<div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-291\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-10.png\" alt=\"\" width=\"544\" height=\"372\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-10.png 544w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-10-300x205.png 300w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-10-65x44.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-10-225x154.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-10-350x239.png 350w\" sizes=\"auto, (max-width: 544px) 100vw, 544px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<\/div>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Notice that the time in the <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> 3 categories of races ( <\/span>rst<span style=\"text-align: initial;font-size: 1em\"> three columns ) are given in seconds whereas the other times are given in minutes<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">There are several functions from di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> packages for performing PCA :<\/span><\/p>\n<ul>\n<li style=\"text-align: justify\">prcomp<span style=\"text-align: initial;font-size: 1em\">() and <\/span>princomp<span style=\"text-align: initial;font-size: 1em\">() from the built-in R stats package <\/span><\/li>\n<li style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">PCA() from FactoMineR package<\/span><\/li>\n<li style=\"text-align: justify\">dudi.pca() from ade4 package<\/li>\n<\/ul>\n<div>\n<p>\u00a0 \u00a0\u00a0princomp() uses the spectral decomposition approach<\/p>\n<p>The functionsprcomp()andPCA() [FactoMineR] use the singular value decom-position (SVD)<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">According to R help, SVD has slightly better numerical accuracy. Therefore, prcomp() is the preferred function<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Let us read the data into R and we use the R function <\/span>prcomp<span style=\"text-align: initial;font-size: 1em\">()<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">runner.pca=prcomp(data,scale=TRUE)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">names(runner.pca)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">[1] &#8220;sdev&#8221;\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 &#8220;rotation&#8221; &#8220;center&#8221;\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 &#8220;scale&#8221;\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 &#8220;x&#8221;<\/span><\/p>\n<\/div>\n<div>\n<p>&nbsp;<\/p>\n<p>&gt; runner.pca$sdev<\/p>\n<p>&nbsp;<\/p>\n<p>[1]\u00a0\u00a0\u00a0\u00a0 2.5733531 0.9368128 0.3991505 0.3522065 0.2826310 0.2607013 0.2154519<\/p>\n<p>[8]\u00a0\u00a0\u00a0\u00a0 0.1503333<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The argument scale=TRUE speci es that the observations are rst standardized and then PCA is performed. This standardization is required since the variables are in di erent units.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">names(runner.pca)<\/p>\n<p style=\"text-align: justify\">[1] &#8220;sdev&#8221;\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 &#8220;rotation&#8221; &#8220;center&#8221;\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 &#8220;scale&#8221;\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 &#8220;x&#8221;<\/p>\n<p>&nbsp;<\/p>\n<p>runner.pca$sdev<\/p>\n<p>&nbsp;<\/p>\n<p>[1]\u00a0\u00a0\u00a0\u00a0 2.5733531 0.9368128 0.3991505 0.3522065 0.2826310 0.2607013 0.2154519<\/p>\n<p>[8]\u00a0\u00a0\u00a0\u00a0 0.1503333<\/p>\n<p>&nbsp;<\/p>\n<p>When we extract the output sdev from the R object runner.pca we get the square roots of the eigen values of the correlation matrix corresponding to the data<\/p>\n<p>&nbsp;<\/p>\n<p>runner.pca$rotation<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The columns given by extracting the output rotation from the R object run-ner.pca are the eigen vectors corresponding to the eigen values<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">So to obtain the <\/span>eigen values<span style=\"text-align: initial;font-size: 1em\"> we square the &#8216;<\/span>sdev<span style=\"text-align: initial;font-size: 1em\">&#8216;<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">We also obtain the percentage of variability explained by the di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> principal components and the cumulative percentage of variability explained from the formula only.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\"># Eigenvalues\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0eig &lt;- (runner.pca$sdev)^2<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\"># Variances in percentage\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 variance &lt;- eig*100\/sum(eig)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\"># Cumulative variances\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 cumvar &lt;- cumsum(variance)<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">runner.eig&lt;- data.frame(eig = eig, variance = variance, cumvariance = cumvar) runner.eig<\/span><\/p>\n<\/div>\n<div>\n<table>\n<tbody>\n<tr>\n<td>eig<\/td>\n<td>variance<\/td>\n<td>cumvariance<\/td>\n<\/tr>\n<tr>\n<td>6.62214613<\/td>\n<td>82.7768266<\/td>\n<td>82.77683<\/td>\n<\/tr>\n<tr>\n<td>0.87761829<\/td>\n<td>10.9702287<\/td>\n<td>93.74706<\/td>\n<\/tr>\n<tr>\n<td>0.15932114<\/td>\n<td>1.9915143<\/td>\n<td>95.73857<\/td>\n<\/tr>\n<tr>\n<td>0.12404939<\/td>\n<td>1.5506173<\/td>\n<td>97.28919<\/td>\n<\/tr>\n<tr>\n<td>0.07988027<\/td>\n<td>0.9985034<\/td>\n<td>98.28769<\/td>\n<\/tr>\n<tr>\n<td>0.06796515<\/td>\n<td>0.8495644<\/td>\n<td>99.13725<\/td>\n<\/tr>\n<tr>\n<td>0.04641953<\/td>\n<td>0.5802441<\/td>\n<td>99.71750<\/td>\n<\/tr>\n<tr>\n<td>0.02260010<\/td>\n<td>0.2825012<\/td>\n<td>100.00000<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<div><strong><span style=\"font-size: 1em;text-align: initial\">\u00a0 \u00a0 Scree Plot Using Base Graphics<\/span><\/strong><\/div>\n<div><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 To draw the Scree plot we can use the base graphics functions like barplot(), lines() <\/span>etc.<\/div>\n<div><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 The R code is as follows<\/span><\/div>\n<div><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 barplot(runner.eig[, 2], names.arg=1:nrow(runner.eig), main = &#8220;Variances&#8221;,<\/span><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 xlab = &#8220;Principal Components&#8221;,<\/span><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 ylab = &#8220;Percentage of variances&#8221;,<\/span><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 col =&#8221;steelblue&#8221;)<\/span><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 # Add connected line segments to the plot<\/span><\/div>\n<div><span style=\"text-align: initial;font-size: 1em\">\u00a0 \u00a0 lines(x = 1:nrow(runner.eig), runner.eig[, 2], type=&#8221;b&#8221;, pch=19, col = &#8220;red&#8221;)<\/span><\/div>\n<div><\/div>\n<div><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-292 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11.png\" alt=\"\" width=\"481\" height=\"480\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11.png 481w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11-150x150.png 150w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11-300x300.png 300w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11-65x65.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11-225x225.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-11-350x349.png 350w\" sizes=\"auto, (max-width: 481px) 100vw, 481px\" \/><\/div>\n<div>\n<p>\u00a0 \u00a0 <strong>Note that:<\/strong><\/p>\n<ul>\n<li>The 2 principal components retain 94% of original data variability<\/li>\n<li>Instead of 8 original variables<span style=\"text-align: initial;font-size: 1em\"> we may work with 2 principal components<\/span><\/li>\n<\/ul>\n<p><strong>We can achieve <\/strong>signi<strong> cant Reduction in Dimension by PCA<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">There are several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries which aid in PCA. The package <\/span>factoextra<span style=\"text-align: initial;font-size: 1em\"> is used for the visualization of <\/span>theprincipal<span style=\"text-align: initial;font-size: 1em\"> component analysis results.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">factoextra can be installed and loaded as follow :<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">install.packages(&#8220;<\/span>devtools<span style=\"text-align: initial;font-size: 1em\">&#8220;)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">devtools::install_github(&#8220;kassambara\/factoextra&#8221;)<\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">#\u00a0\u00a0 load <\/span><\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">library(&#8220;factoextra&#8221;)<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong style=\"text-align: initial;font-size: 1em\">Scree plot using Factoextra:<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">The R code is <\/span>fviz screeplot<span style=\"text-align: initial;font-size: 1em\">()<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">fviz_screeplot(runner.pca, ncp=10)<\/span><\/p>\n<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-293 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-12.png\" alt=\"\" width=\"524\" height=\"455\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-12.png 524w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-12-300x260.png 300w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-12-65x56.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-12-225x195.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-12-350x304.png 350w\" sizes=\"auto, (max-width: 524px) 100vw, 524px\" \/><\/p>\n<div>\n<p>&nbsp;<\/p>\n<p>Example<strong><span style=\"text-align: initial;font-size: 1em\"> where PCA is not worthwhile<\/span><\/strong><\/p>\n<\/div>\n<div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-294\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-13.png\" alt=\"\" width=\"649\" height=\"519\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-13.png 649w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-13-300x240.png 300w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-13-65x52.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-13-225x180.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-13-350x280.png 350w\" sizes=\"auto, (max-width: 649px) 100vw, 649px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>Note that:<\/p>\n<p>&nbsp;<\/p>\n<p>The\u00a0 first 5 principal components together explain 93% of the total variability<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">So we do not achieve any significant reduction in dimension by PCA for this problem. This may be attributed to the fact that the variables may have very low correlations amongst themselves.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">Scree Plot<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">par(mar=c(5.1,4.85,4.1,2.1))<\/span><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">plot(1:6,eigen.values, type=&#8221;o&#8221;, main=&#8221;Scree Plot&#8221;, xlab=&#8221;Index&#8221;, ylab=expression(lambda[j]), cex.lab=1.5, col=&#8221;dark red&#8221;,lwd=2)<\/span><\/p>\n<\/div>\n<div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-295 aligncenter\" src=\"http:\/\/statp11.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-14.png\" alt=\"\" width=\"536\" height=\"474\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-14.png 536w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-14-300x265.png 300w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-14-65x57.png 65w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-14-225x199.png 225w, https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-content\/uploads\/sites\/139\/2018\/12\/Untitled-14-350x310.png 350w\" sizes=\"auto, (max-width: 536px) 100vw, 536px\" \/><\/p>\n<p style=\"text-align: justify\">Even from the Scree plot we see that the rst bend in the curve occurs for index 2, however at such a point the jump is not signi cantly close to 0. This bend and signi cant jump can be seen almost at index 5<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong><span style=\"text-align: initial;font-size: 1em\">SUMMARY<\/span><\/strong><\/p>\n<ul>\n<li style=\"text-align: justify\"><span style=\"text-align: initial;font-size: 1em\">In R we can perform Principal Component Analysis from the <\/span>rst princi-ples<span style=\"text-align: initial;font-size: 1em\"> or using several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries<\/span><\/li>\n<li style=\"text-align: justify\">In R we can perform Principal Component Analysis from the rst princi-ples<span style=\"text-align: initial;font-size: 1em\"> or using several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries<\/span><\/li>\n<li style=\"text-align: justify\">In R we can perform Principal Component Analysis from the rst princi-ples<span style=\"text-align: initial;font-size: 1em\"> or using several inbuilt R functions under di <\/span>erent<span style=\"text-align: initial;font-size: 1em\"> libraries<\/span><\/li>\n<\/ul>\n<\/div>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td><strong>you can view video on Principal Component Analysis using R<\/strong><\/td>\n<td><a href=\"https:\/\/youtu.be\/W6oaWajCIG8\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-120\" src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"\" width=\"36\" height=\"36\" \/><\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><strong>References<\/strong><\/p>\n<ul>\n<li>R.A.Johnson &amp; D.W. Wichern, Applied Multivariate Statistical Analysis, Pearson<\/li>\n<li>T.W. Anderson, An Introduction to Multivariate Analysis, John Wiley<\/li>\n<li>G.A.F. Seber, Multivariate Observations, John Wiley<\/li>\n<li>N.C. Giri, Multivariate Statistical Inference, Academic Press<\/li>\n<\/ul>\n","protected":false},"author":3,"menu_order":19,"template":"","meta":{"_acf_changed":false,"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":["prof-sumitra-purkayastha"],"pb_section_license":""},"chapter-type":[],"contributor":[59],"license":[],"class_list":["post-276","chapter","type-chapter","status-publish","hentry","contributor-prof-sumitra-purkayastha"],"part":3,"_links":{"self":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/pressbooks\/v2\/chapters\/276","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/wp\/v2\/users\/3"}],"version-history":[{"count":5,"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/pressbooks\/v2\/chapters\/276\/revisions"}],"predecessor-version":[{"id":296,"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/pressbooks\/v2\/chapters\/276\/revisions\/296"}],"part":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/pressbooks\/v2\/parts\/3"}],"metadata":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/pressbooks\/v2\/chapters\/276\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/wp\/v2\/media?parent=276"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/pressbooks\/v2\/chapter-type?post=276"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/wp\/v2\/contributor?post=276"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/statp11\/wp-json\/wp\/v2\/license?post=276"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}