Part of being in graduate school requires you to be involved in some sort of research. It may be assisting on a project led by a professor, developing your own project, or just sitting in on research meetings.
Since I am finishing up my second quarter in Electrical and Computer Engineering graduate school, I am entering the research phase. Funnily enough, the research I have chosen to be a part of has something to do with smartphones. I will describe it how below. (Note – this blog post is rather long and lightly technical. Just lightly!)
The hot topic these days in computer science, statistics, and to some extent computer engineering is “Machine Learning”. This area of research sits at the intersection of the above fields of study. Essentially, the researchers in this field ask two questions:
- Can we predict certain phenomena using tremendously large data sets?
- Can we ‘teach’ a computer to behave in a certain way, and adapt its behavior to changes in its environment based on incoming data?
These two questions open tremendous horizons for the researcher. From a commercial point of view, there’s great benefit here. Can a company predict what a potential customer will want to buy (Amazon) based on all the information it has? Can the company predict what the customer will want to watch (Netflix)?
From a non commercial perspective there is benefit as well. Can Google label the billions of images it indexes based on content automatically? This would allow users to search for images without human-hours needed to categorize. Can a smart car learn and adapt its behavior based on circumstances?
One of the challenges with this problem is related to computational power. To go from input to proper output, for categorizing an image for example, requires the input to go through many layers of ‘neural network’. Basically each data point has to traverse an oftentimes large weighted graph to be categorized properly. And these graphs take up a lot of memory – 200MB at a minimum. If you consider a typical smartphone to have about 512MB – 2GB of memory, those 200+MB take up 10-50% of all available memory – certainly a hefty amount. Not only that, but accessing this memory is a big time battery drain.
Essentially my research revolves around techniques for compressing these ‘neural networks’ by very large factors (basically we want to scale it down to several MB, or 2-5% of the original size) while keeping accuracy. With sizes like this it becomes very easy to load these networks on smartphones, allowing a user for example to take a picture and have his or her phone instantly identify what that picture is, without having to go to the cloud (sending the picture across the internet).
And funny enough, this research is related to the very thing that I am trying to use less!