Learning Deep Learning
Introduction
This reading list is an on-going personal challenge I’m about to start again. The challenge is to digest one paper a day in the morning (a.k.a eating a paper as breakfast). Over time I found that one can really really read a lot in an hour. And it’s a perfect way of starting a new day with some new interesting knowledge.
In general,
The old version of this post is now hosted at here. I will preserve this legacy version only for lookup reasons.
For Novice
If you have no idea about Machine Learning and Scientific Computing, I suggest you learn the following materials while you are reading Machine Learning or Deep Learning books. You don’t have to master these materials, but a basic understanding is essential. It’s hard to open a meaningful conversation if the person has no idea about matrix or single variable calculus.
Title | Author or Source | Tags |
---|---|---|
Introduction to Algorithms | Erik Demaine and Srinivas Devadas | |
Single Variable Calculus | David Jerison | |
Multivariable Calculus | Denis Auroux | |
Differential Equations | Arthur Mattuck, Haynes Miller, Jeremy Orloff, John Lewis | |
Linear Algebra | Gilbert Strang |
Theory of Computation, Learning Theory, Neuroscience, etc
Fundamentals of Deep Learning
Title | Author or Source | Tags |
---|---|---|
Deep Learning in Neural Networks: An Overview | Jürgen Schmidhuber | |
Deep Learning Book | Yoshua Bengio, Ian Goodfellow and Aaron Courville | |
Learning Deep Architectures for AI | Yoshua Bengio | |
Representation Learning: A Review and New Perspectives | Yoshua Bengio, Aaron Courville, Pascal Vincent | |
Reading lists for new MILA students | MILA Lab, University of Montreal | |
Tutorial on Variational Autoencoders | Carl Doersch |
Tutorials, Practical Guides, and Useful Software
Title | Author or Source | Tags |
---|---|---|
Machine Learning | Andrew Ng | |
Neural Networks for Machine Learning | Geoffrey Hinton | |
Deep Learning Tutorial | LISA Lab, University of Montreal | |
Unsupervised Feature Learning and Deep Learning Tutorial | AI Lab, Stanford University | |
CS231n: Convolutional Neural Networks for Visual Recognition | Stanford University | |
CS224d: Deep Learning for Natural Language Processing | Stanford University | |
Theano | LISA Lab, University of Montreal | |
PyLearn2 | LISA Lab, University of Montreal | |
Caffe | Berkeley Vision and Learning Center (BVLC) and community contributor Yangqing Jia | |
Torch 7 | active contributors | |
neon | Nervana | |
cuDNN | NVIDIA | |
ConvNetJS | Andrej Karpathy | |
DeepLearning4j | ||
Chainer: Neural network framework | Preferred Networks, Inc | |
Blocks | LISA Lab, University of Montreal | |
Fuel | LISA Lab, University of Montreal | |
Brainstorm | IDSIA, Switzerland | |
Keras | fchollet and active contributors | |
Lasagne | Lasagne |
Literature in Deep Learning and Feature Learning
Deep Learning is a fast-moving community. Therefore the line between “Recent Advances” and “Literature that matter” is kind of blurred. Here I collected articles that are either introducing fundamental algorithms, techniques or highly cited by the community.
Title | Author or Source | Tags |
---|---|---|
Automatic Speech Recognition - A Deep Learning Approach | Dong Yu and Li Deng (Published by Springer, no Open Access) | |
Backpropagation Applied to Handwritten Zip Code Recognition | Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. D. Jackel | |
Comparison of Training Methods for Deep Neural Networks | Patrick O. Glauner | |
Deep Learning | Yann LeCun, Yoshua Bengio, Geoffrey Hinton. (NO FREE COPY AVAILABLE) | |
Distributed Representations of Words and Phrases and their Compositionality | Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean | |
Efficient Estimation of Word Representations in Vector Space | Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean | |
Efficient Large Scale Video Classification | Balakrishnan Varadarajan, George Toderici, Sudheendra Vijayanarasimhan, Apostol Natsev | |
Foundations and Trends in Signal Processing: DEEP LEARNING — Methods and Applications | Li Deng and Dong Yu | |
From Frequency to Meaning: Vector Space Models of Semantics | Peter D. Turney and Patrick Pantel | |
LSTM: A Search Space Odyssey | Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R. Steunebrink, Jürgen Schmidhuber | |
Supervised Sequence Labelling with Recurrent Neural Networks | Alex Graves |
Recent Must-Read Advances in Deep Learning
Contacts and Suggestions
Yuhuang Hu
Email: duguyue100@gmail.com