<div><h1>
Principal component analysis
</h1>
<p>
In this post I will explain the basics of<a href="https://en.wikipedia.org/wiki/Principal_component_analysis" target="_blank" rel="noreferrer" >Principal Component Analysis (PCA)</a>, when and why to use it, and how to implement a basic example. This post assumes the reader has taken at leastone semester of statistics and linear algebra.</p></div><div><h2>What is Principal Component Analysis</h2></div><div><p>
Principal Component Analysis (PCA) is a statistical technique that allows you to extract information from a large dataset by means of breaking the dataset down into smaller components. This is useful when working with a large dataset that has a lot of features. PCA is used by statisticians everywhere - from 3D
rendering to image compression - because it is one of the
easiest ways to extract relevant information from datasets that contain complex information.<br /><br /> You can think of PCA as the<a href="https://en.wikipedia.org/wiki/SparkNotes"
target="_blank" rel="noreferrer">Sparknotes</a>
version of a lengthy novel. For some, finding the time to read a
lengthy novel may be a luxury they can&#39;t afford. Instead,
wouldn&#39;t it be nice to have a summary of the book that covers
the most important aspects of the story in just a couple pages?<br /><br /></p></div>\n<div>
              <h2 >How does it work?</h2>
            </div>
            <div >
              <p>
                Unfortunetly for us, data does not speak English. So in order to
                find meaning in our data we will have to use mathematics. A
                couple questions you might be asking yourself is, how does PCA
                even understand what is important in our data? And then, how do
                we even quanitfy the information that lives within our data? The
                key to PCA is<a
                  href="https://en.wikipedia.org/wiki/Variance"
                  target="_blank"
                  rel="noreferrer">variance</a>.<br /><br />The greater the variance in our data, the more information we
can extract. Why? Lets play a game for a moment. The game is
simple, I have picked out three books and painted the covers of
each, so you do not know the titles. I present you with a list
that states the title and number of pages in each book. Your job
is to try and match the titles to the books, based on the number
of pages.
</p><div className="divtable"><table ">
<tr >
<th>Title</th>
<th># of pages</th>
</>
<tr >
<td >A Game of Thrones</td>
<td >694</td>
</tr>
<tr >
<td >The Cat in the Hat</td>
<td >61</td>
</tr>
<tr >
<td >The Lord of the Rings</td>
                    <td >1178</td>
                  </tr>
                </table>
              </div>
              <div>
                <p >
                  Now that we have our list, here are the books:
                </p>
              </div>
              <div >
                <Image
                  src="/BookTransparent.png"
                  alt="books"
                  width={200}
                  height={200}
                />
              </div>
              <div >
                <p>
                  It is easy to see that [A] is <i>The Lord of the Rings</i>,
                  [B] is <i>A Game of Thrones</i>, and [C] is
                  <i>The Cat in the Hat</i> <br />
                  <br />
                  Now lets try a different set of books. Here is your list:
                </p>
              </div>
              <div >
                <table >
                  <tr >
                    <th>Title</th>
                    <th># of pages</th>
                  </tr>
                  <tr >
                    <td >The Epic of Gilgamesh</td>
                    <td >128</td>
                  </tr>
                  <tr >
                    <td >The Old Man and the Sea</td>
                    <td c>127</td>
                  </tr>
                  <tr >
                    <td >Animal Farm</td>
                    <td >130</td>
                  </tr>
                </table>
              </div>
              <div>
                <p>And here are the books.</p>
              </div>
              <div >
                <Image
                  src="/BooksTransparent2.png"
                  alt="books2"
                  width={300}
                  height={300}
                />
              </div>
              <div>
                <p>
                  Can you guess which book is which? It is a lot harder when the
                  books are similar in length. In the first example we had no
                  issues identifying the books, becuase the book lengths
                  <i>varied</i> a lot.
                </p>
              </div>
              <div >
                <p>
                  This is what I meant earlier when I said that data with more
                  variance, has more information. So through PCA, we can
                  quanitfy the information that lives within our data through{" "}
                  <i>variance</i>.{" "}
                </p>
              </div>
              <div>
                <p>
                  Lets take the first example one step further. Suppose I also
                  give you the weight of the books.
                </p>
              </div>
              <div >
                <table>
                  <tr >
                    <th>Title</th>
                    <th># of pages</th>
                    <th>Weight (oz)</th>
                  </tr>
                  <tr >
                    <td >A Game of Thrones</td>
                    <td >694</td>
                    <td >11.4</td>
                  </tr>
                  <tr >
                    <td>The Cat in the Hat</td>
                    <td>61</td>
                    <td >10.9</td>
                  </tr>
                  <tr>
                    <td >The Lord of the Rings</td>
                    <td >1178</td>
                    <td >11.3</td>
                  </tr>
                </table>
              </div>
              <div >
                <p>
                  Now that we have added weight, does your guessing strategy
                  change? Because the <i>variance</i> of the weights are so
                  small, we would still mostly rely on the number of pages to
                  make our guess. We intrinsically know that we need to place
                  more emphasis on the number of pages than on the weight when
                  making our guess, so in theory we could eliminate the weight
                  column and we would hardly lose any useful information. This
                  is the essence of PCA, minimizing the complexity of the data,
                  while maximizing the amount of information. Ok, but how do we
                  quantify it?
                </p>
              </div>
            </div>\n

The utilization of Principal Component Analysis (PCA) has become commonplace in modern data analysis, yet its inner workings are often shrouded in mystery. The aim of this post is to dispel the magic behind this black box.

Blog posts