In this post, I recommend some helpful resources for learning deep neural networks based on my learning experience.

lectures & textbook

The standford cs231n is a good start to learn CNNs. The cs224n is about natural language processing with deep learning.

After finish cs231n and cs224n, I would recommend the MIT deep learning book.

If you are interested in reinforcement learning, David Silver is a good teacher. The videos of his RL cource is available here

blogs

Blogs usually simplified concepts to make them easier to understand. Thus, they are more friendly to beginners.

Here are some high-quality blogs I have read about CNNs.

A Beginner’s Guide To Understanding Convolutional Neural Networks. PS: the author was only 19 when he wrote this high-quality blog. Really impressive.

A 2017 Guide to Semantic Segmentation with Deep Learning

A Year in Computer Vision is a good report about DNNs in CV.

Review of Deep Learning Algorithms for Object Detection

important models

two survey papers I like:
Recent Advances in Recurrent Neural Networks
Recent Advances in Convolution Neural Networks

There is a github repo that collects the state-of-the-art results for machine learning problems.

DNN Genealogy, an interactive visualization tool developed by our group, summarizes the milestone models in the development of DNNs.

DNN genealogy

The Caffe Model Zoo is also a good place to look for DNN models.

Framework

Don’t start with tensorflow!
Even though tensorflow is the most famous and popular machine learning framework, it is definitely not friendly to beginners.

I would recommend Keras and Pytorch.

ranking

I like Keras because:
a) it supports multiple backend engines, including TensorFlow, CNTK, Theano, and MXNet;
b) it offers consistent and simple APIs that minimizes the number of actions required for common use cases.

Keras offers offical examples, implementations of popular models, and a number of pre-trained models.

Also, in the community, almost all the state-of-the-art models have their Keras versions. For example, titu1994 reimplements many famouse DNNs using Keras, including NasNet, non-local NN, mobileNet. So, you can easily play with the state-of-art model using Keras.