Neural Network Tricks In this notebook, I’ll show the effects of various techniques (“tricks”) used to improve the performance of neural networks. Most of them come from the (LeCun, Bottou, Orr and Muller, 2012) paper. Previously, we built a basic neural network in the “Backprop Exercise” notebook. Here, I’ll use...

Backpropagation Exercise Let’s implement the backpropagation algorithm to train a simple neural network with a single hidden layer. What is backpropagation? Previously,the stochastic gradient decent rule for training perceptrons was derived, which can be thought of as a single-layer (linear) neural network. The backpropagation algorithm is a learning rule that...

Gabor Filters In this notebook, I will describe what Gabor Filters are and demonstrate some of their uses. What are Gabor filters? Gabor filter, named after the physicist and electrical engineer Dennis Gabor, is a linear filter used for edge detection in image processing. (Trivia: Dennis Gabor invented holography and...

Perceptron In this notebook, I will go through how to train a perceptron for binary classification problems. What is a perceptron? Perceptron is a artificial neural network whose learning was invented by Frak Rosenblatt in 1957. According to wikipedia, “In a 1958 press conference organized by the US Navy, Rosenblatt...