Dimensionality Reduction with PCA (Principal Component Analysis)
In this blog, we will be discussing the following topics:
// TODO: fill this post the draft
Introduction
Dealing with data comes with own its whole bunch of picularities. Dimensionality reduction was one such pecularity to me. I couldn't wrap my head around the idea of retaining the same level/"usefullness" of data while at the same time reducing the size of it. How could one, reduce the data size while maintaining the "jist" of it.
As my first introduction to PCA was in a Data Science Bootcamp, the instructors didn't bother to go into the mathematically details and I was too uninterested in the dirty math to look it up myself. The result was my poor application capability to apply it almost on any dataset without having the slightest of idea if PCA is even required there. You import a few sckit learn functions, apply them on pandas dataframes and it was done, AFAIK.
Thanksfully, I had the time to look it up carefully again as it now became part of my university course curriculm. This time, I was able to wrap my head it, so much so that I am willing to take the risk of writing this blog, so you can also wrap your head around it :). Honestly, the concept in itself is quiet ingenious if you understand it.
So, let's start. Keep a jupyter notebook open if you want to follow along.
The Need
The understanding of the term "Dimensionality Reduction" should explain the need for it. Any tabular dataset is composed of features/dimensions and datapoints. While datapoints themselves represent individual entries, the features explain various properties of a data point.
For example, you are running the health survey in a school, the various data of students would comprise the datapoints while their individual metrics like height, weight, age etc would be features.
It is important to note that generation of these data is based upon a small probabalictic caveat that the datapoints themselves follow IID (Independent and Identically Distributed) assumptions, i.e.
-
Independent : Generation of one data point is independent of other generated data points.
-
Identically Distributed : Since every data point is randomly generated, it gets bounded by the laws of probability, meaning there is some underlying hidden process by which the data is generated. This assumption ensures that every data point is coming from the same underlying distribution.
Remove this later:
[O(n^3)]
$$ e^{i\pi} + 1 = 0 $$