Linear Algebraic Methods in Data Science and Neural Networks (thesis)

Abstract

This thesis is about some of the methods and concepts of linear algebra that are particularly helpful for data analysis. After a brief review of some linear algebra concepts in chapter 1, the second chapter of the thesis centers around the singular value decomposition (SVD) which expresses any matrix A as a product of an orthogonal matrix, a diagonal matrix, and another orthogonal matrix. Understanding the SVD requires understanding the properties of symmetric matrices, which are explained first. Chapter 3 focuses on the applications of the SVD. It begins with using the SVD for low-rank approximation, and then explores how the SVD is applied in principle component analysis. Chapter 4 introduces neural networks, a machine learning architecture useful for image recognition among other applications. It introduces the structure of a neural network in linear algebraic notation; one of the main goals of chapter 4 is to reinforce the idea that neural networks can be seen as compositions of matrix transformations with non-linear activation functions. We then introduce how the parameters in a neural network are optimized. Chapter 5 deals with convolutional neural networks. It also focuses heavily on circulant matrices, and the relationship between convolution and circulant matrices. Understanding the properties of circulant matrices will be instrumental in understanding the benefits of convolution. We finish the chapter by showing how PCA can be used as a data pre-processing tool before running the data through a convolutional neural network.