GoogLeNet

The GoogLeNet published in 2015 presented an effective and simple idea. Instead of applying a 3x3 or 5x5 convolution layer directly, in later layers, first apply a 1x1 convolution layer and then apply the real convolution layer. The 1x1 convolution average all depth values of each pixel and reduce the depth to one. So the 1x1 convolution is ...

Read More

Recurrent neural networks (RNN)

Different from regular neural networks (including deep neural networks), recurrent neural networks (RNN) provides a representation that connects a sequence of inputs. It could a time sequence, a DNA sequence, or any other sequence. A regular NN takes one input (a vector) at a time and the output responds to the input alone, not the previous inputs ...

Read More

Review on “Mastering the game of Go with deep neural networks and tree search” (AlphaGo)

Google DeepMind's AlphaGo won 5-game challenge series 4-1 overall in Seoul against the world-class player Lee Se-dol, the top Go player in the world over the past decade. The video streams of the matches are on Youtube. The paper "Mastering the game of Go with deep neural networks and tree search" published in Nature on 28th January 2016, ...

Read More

Restricted Boltzmann Machine (RBM)

[latexpage] 1. Background Learning restricted Boltzmann machine (RBM) is not easy for people who don't have much knowledge in statistics. Even for people who have learned basic machine learning/data mining, it is not very straight forward. The students in my Neural Network and Deep Learning class struggled quite a bit in understanding the ...

Read More