DGY

Learning Deep Learning

There are lots of awesome reading lists or posts that summarized materials related to Deep Learning. So why would I commit another one? Well, the primary objective is to develop a complete reading list that allows readers to build a solid academic and practical background of Deep Learning. And this list is developed while I’m preparing my Deep Learning workshop. My research is related to Deep Neural Networks (DNNs) in general. Hence, this post tends to summary contributions in DNNs instead of generative models.

For Novice

If you have no idea about Machine Learning and Scientific Computing, I suggest you learn the following materials while you are reading Machine Learning or Deep Learning books. You don’t have to master these materials, but a basic understanding is essential. It’s hard to open a meaningful conversation if the person has no idea about matrix or single variable calculus.

Title Author or Source Tags
Introduction to Algorithms Erik Demaine and Srinivas Devadas  
Single Variable Calculus David Jerison  
Multivariable Calculus Denis Auroux  
Differential Equations Arthur Mattuck, Haynes Miller, Jeremy Orloff, John Lewis  
Linear Algebra Gilbert Strang  

Theory of Computation, Learning Theory, Neuroscience, etc

Title Author or Source Tags
Introduction to the Theory of Computation Michael Sipser  
Artificial Intelligence: A Modern Approach Stuart Russell and Peter Norvig  
Pattern Recognition and Machine Learning Christopher Bishop  
Machine Learning: A probabilistic perspective Kevin Patrick Murphy  
CS229 Machine Learning Course Materials Andrew Ng at Stanford University  
Reinforcement Learning: An Introduction Richard S. Sutton and Andrew G. Barto  
Probabilistic Graphical Models: Principles and Techniques Daphne Koller and Nir Friedman  
Convex Optimization Stephen Boyd and Lieven Vandenberghe  
An Introduction to Statistical Learning with application in R Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani  
Neuronal Dynamics: From single neurons to networks and models of cognition Wulfram Gerstner, Werner M. Kistler, Richard Naud and Liam Paninski  
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems Peter Dayan and Laurence F. Abbott  
Michael I. Jordan Reading List of Machine Learning Hacker News  

Fundamentals of Deep Learning

Title Author or Source Tags
Deep Learning in Neural Networks: An Overview Jürgen Schmidhuber  
Deep Learning Book Yoshua Bengio, Ian Goodfellow and Aaron Courville  
Learning Deep Architectures for AI Yoshua Bengio  
Representation Learning: A Review and New Perspectives Yoshua Bengio, Aaron Courville, Pascal Vincent  
Reading lists for new MILA students MILA Lab, University of Montreal  
Tutorial on Variational Autoencoders Carl Doersch  

Tutorials, Practical Guides, and Useful Software

Title Author or Source Tags
Machine Learning Andrew Ng  
Neural Networks for Machine Learning Geoffrey Hinton  
Deep Learning Tutorial LISA Lab, University of Montreal  
Unsupervised Feature Learning and Deep Learning Tutorial AI Lab, Stanford University  
CS231n: Convolutional Neural Networks for Visual Recognition Stanford University  
CS224d: Deep Learning for Natural Language Processing Stanford University  
Theano LISA Lab, University of Montreal  
PyLearn2 LISA Lab, University of Montreal  
Caffe Berkeley Vision and Learning Center (BVLC) and community contributor Yangqing Jia  
Torch 7 active contributors  
neon Nervana  
cuDNN NVIDIA  
ConvNetJS Andrej Karpathy  
DeepLearning4j    
Chainer: Neural network framework Preferred Networks, Inc  
Blocks LISA Lab, University of Montreal  
Fuel LISA Lab, University of Montreal  
Brainstorm IDSIA, Switzerland  
Keras fchollet and active contributors  
Lasagne Lasagne  

Literature in Deep Learning and Feature Learning

Deep Learning is a fast-moving community. Therefore the line between “Recent Advances” and “Literature that matter” is kind of blurred. Here I collected articles that are either introducing fundamental algorithms, techniques or highly cited by the community.

Title Author or Source Tags
Automatic Speech Recognition - A Deep Learning Approach Dong Yu and Li Deng (Published by Springer, no Open Access)  
Backpropagation Applied to Handwritten Zip Code Recognition Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. D. Jackel  
Comparison of Training Methods for Deep Neural Networks Patrick O. Glauner  
Deep Learning Yann LeCun, Yoshua Bengio, Geoffrey Hinton. (NO FREE COPY AVAILABLE)  
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean  
Efficient Estimation of Word Representations in Vector Space Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean  
Efficient Large Scale Video Classification Balakrishnan Varadarajan, George Toderici, Sudheendra Vijayanarasimhan, Apostol Natsev  
Foundations and Trends in Signal Processing: DEEP LEARNING — Methods and Applications Li Deng and Dong Yu  
From Frequency to Meaning: Vector Space Models of Semantics Peter D. Turney and Patrick Pantel  
LSTM: A Search Space Odyssey Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R. Steunebrink, Jürgen Schmidhuber  
Supervised Sequence Labelling with Recurrent Neural Networks Alex Graves  

Recent Must-Read Advances in Deep Learning

Most of papers here are produced in 2014 and after. Survey papers or review papers are not included.

Title Author or Source Tags
A Convolutional Attention Network for Extreme Summarization of Source Code Miltiadis Allamanis, Hao Peng, Charles Sutton  
A Deep Bag-of-Features Model for Music Auto-Tagging Juhan Nam, Jorge Herrera, Kyogu Lee  
A Deep Generative Deconvolutional Image Model Yunchen Pu, Xin Yuan, Andrew Stevens, Chunyuan Li, Lawrence Carin  
A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games Felix Leibfried, Nate Kushman, Katja Hofmann  
A Deep Neural Network Compression Pipeline: Pruning, Quantization, Huffman Encoding Song Han, Huizi Mao, William J. Dally  
A Deep Pyramid Deformable Part Model for Face Detection Rajeev Ranjan, Vishal M. Patel, Rama Chellappa  
A Deep Siamese Network for Scene Detection in Broadcast Videos Lorenzo Baraldi, Costantino Grana, Rita Cucchiara  
A Hierarchical Approach for Generating Descriptive Image Paragraphs Jonathan Krause, Justin Johnson, Ranjay Krishna, Li Fei-Fei  
A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob G. Simonsen, Jian-Yun Nie  
A Large-Scale Car Dataset for Fine-Grained Categorization and Verification Linjie Yang, Ping Luo, Chen Change Loy, Xiaoou Tang  
A Lightened CNN for Deep Face Representation Xiang Wu, Ran He, Zhenan Sun  
A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction Thomas Wiatowski, Helmut Bölcskei  
A Multi-scale Multiple Instance Video Description Network Huijuan Xu, Subhashini Venugopalan, Vasili Ramanishka, Marcus Rohrbach, Kate Saenko  
A Neural Attention Model for Abstractive Sentence Summarization Alexander M. Rush, Sumit Chopra, Jason Weston  
A Recurrent Latent Variable Model for Sequential Data Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, Yoshua Bengio  
A Restricted Visual Turing Test for Deep Scene and Event Understanding Hang Qi, Tianfu Wu, Mun-Wai Lee, Song-Chun Zhu  
A Self-Driving Robot Using Deep Convolutional Neural Networks on Neuromorphic Hardware Tiffany Hwu, Jacob Isbell, Nicolas Oros, Jeffrey Krichmar  
A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification Ye Zhang, Byron Wallace  
ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering Kan Chen, Jiang Wang, Liang-Chieh Chen, Haoyuan Gao, Wei Xu, Ram Nevatia  
Accelerating Very Deep Convolutional Networks for Classification and Detection Xiangyu Zhang, Jianhua Zou, Kaiming He, Jian Sun  
Accurate Image Super-Resolution Using Very Deep Convolutional Networks Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee  
Action Recognition using Visual Attention Shikhar Sharma, Ryan Kiros, Ruslan Salakhutdinov  
Action Recognition With Trajectory-Pooled Deep-Convolutional Descriptors Limin Wang, Yu Qiao, Xiaoou Tang  
Action-Conditional Video Prediction using Deep Networks in Atari Games Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard Lewis, Satinder Singh  
Active Object Localization with Deep Reinforcement Learning Juan C. Caicedo, Svetlana Lazebnik  
adaQN: An Adaptive Quasi-Newton Algorithm for Training RNNs Nitish Shirish Keskar, Albert S. Berahas  
AdaScan: Adaptive Scan Pooling in Deep Convolutional Neural Networks for Human Action Recognition in Videos Amlan Kar, Nishant Rai, Karan Sikka, Gaurav Sharma  
Adding Gradient Noise Improves Learning for Very Deep Networks Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens  
Adversarial Autoencoders Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow  
Adversarial Manipulation of Deep Representations Sara Sabour, Yanshuai Cao, Fartash Faghri, David J. Fleet  
All you need is a good init Dmytro Mishkin, Jiri Matas  
An Efficient Approach to Boosting Performance of Deep Spiking Network Training Seongsik Park, Sang-gil Lee, Hyunha Nam, Sungroh Yoon  
An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition Baoguang Shi, Xiang Bai, Cong Yao  
An Uncertain Future: Forecasting from Static Images using Variational Autoencoders Jacob Walker, Carl Doersch, Abhinav Gupta, Martial Hebert  
Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering Xiaoqiang Zhou, Baotian Hu, Qingcai Chen, Buzhou Tang, Xiaolong Wang  
Anticipating the future by watching unlabeled video Carl Vondrick, Hamed Pirsiavash, Antonio Torralba  
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering Haoyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, Wei Xu  
Artificial Neural Networks Applied to Taxi Destination Prediction Alexandre de Brébisson, Étienne Simon, Alex Auvolat, Pascal Vincent, Yoshua Bengio  
Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering Huijuan Xu, Kate Saenko  
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, Richard Socher  
Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources Qi Wu, Peng Wang, Chunhua Shen, Anton van den Hengel, Anthony Dick  
Ask Your Neurons: A Neural-based Approach to Answering Questions about Images Mateusz Malinowski, Marcus Rohrbach, Mario Fritz  
Associative Long Short-Term Memory Ivo Danihelka, Greg Wayne, Benigno Uria, Nal Kalchbrenner, Alex Graves  
Asynchronous Stochastic Gradient Descent with Delay Compensation for Distributed Deep Learning Shuxin Zheng, Qi Meng, Taifeng Wang, Wei Chen, Nenghai Yu, Zhi-Ming Ma, Tie-Yan Liu  
AttentionNet: Aggregating Weak Directions for Accurate Object Detection Donggeun Yoo, Sunggyun Park, Joon-Young Lee, Anthony Paek, In So Kweon  
Attention-Based Models for Speech Recognition Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, Yoshua Bengio  
Attention to Scale: Scale-aware Semantic Image Segmentation Liang-Chieh Chen, Yi Yang, Jiang Wang, Wei Xu, Alan L. Yuille  
Attention with Intention for a Neural Network Conversation Model Kaisheng Yao, Geoffrey Zweig, Baolin Peng  
AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery Izhar Wallach, Michael Dzamba, Abraham Heifets  
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe and Christian Szegedy  
Batch Normalized Recurrent Neural Networks César Laurent, Gabriel Pereyra, Philémon Brakel, Ying Zhang, Yoshua Bengio  
Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding Alex Kendall, Vijay Badrinarayanan, Roberto Cipolla  
Better Computer Go Player with Neural Network and Long-term Prediction Yuandong Tian, Yan Zhu  
Better Exploiting OS-CNNs for Better Event Recognition in Images Limin Wang, Zhe Wang, Sheng Guo, Yu Qiao  
Benchmarking of LSTM Networks Thomas M. Breuel  
Beyond Short Snipets: Deep Networks for Video Classification Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan, Oriol Vinyals, Rajat Monga, George Toderici  
Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video Lionel Pigou, Aäron van den Oord, Sander Dieleman, Mieke Van Herreweghe, Joni Dambre  
Binarized Neural Networks Itay Hubara, Daniel Soudry, Ran El Yaniv  
BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 Matthieu Courbariaux, Yoshua Bengio  
Binding via Reconstruction Clustering Klaus Greff, Rupesh Kumar Srivastava, Jürgen Schmidhuber  
Bottom-up and top-down reasoning with convolutional latent-variable models Peiyun Hu, Deva Ramanan  
Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture Ashesh Jain, Hema S Koppula, Shane Soh, Bharad Raghavan, Avi Singh, Ashutosh Saxena  
Brain-Inspired Deep Networks for Image Aesthetics Assessment Zhangyang Wang, Florin Dolcos, Diane Beck, Shiyu Chang, Thomas S. Huang  
Can Active Memory Replace Attention? Łukasz Kaiser, Samy Bengio  
Character-level Convolutional Networks for Text Classification Xiang Zhang, Junbo Zhao, Yann LeCun  
Compositional Memory for Visual Question Answering Aiwen Jiang, Fang Wang, Fatih Porikli, Yi Li  
Compressing Convolutional Neural Networks Wenlin Chen, James T. Wilson, Stephen Tyree, Kilian Q. Weinberger, Yixin Chen  
Compressing LSTMs into CNNs Krzysztof J. Geras, Abdel-rahman Mohamed, Rich Caruana, Gregor Urban, Shengjie Wang, Ozlem Aslan, Matthai Philipose, Matthew Richardson, Charles Sutton  
Compression of Deep Neural Networks on the Fly Guillaume Soulié, Vincent Gripon, Maëlys Robert  
Confusing Deep Convolution Networks by Relabelling Leigh Robinson, Benjamin Graham  
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation Deepak Pathak, Philipp Krähenbühl, Trevor Darrell  
Continuous control with deep reinforcement learning Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra  
Convergent Learning: Do different neural networks learn the same representations? Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, John Hopcroft  
Convolutional Clustering for Unsupervised Learning Aysegul Dundar, Jonghoon Jin, Eugenio Culurciello  
Convolutional Color Constancy Jonathan T. Barron  
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, Wang-chun Woo  
Convolutional Pose Machines Shih-En Wei, Varun Ramakrishna, Takeo Kanade, Yaser Sheikh  
Convolutional Residual Memory Networks Joel Moniz, Christopher Pal  
DAG-Recurrent Neural Networks For Scene Labeling Bing Shuai, Zhen Zuo, Gang Wang, Bing Wang  
Data-dependent Initializations of Convolutional Neural Networks Philipp Krähenbühl, Carl Doersch, Jeff Donahue, Trevor Darrell  
Data-free parameter pruning for Deep Neural Networks Suraj Srinivas, R. Venkatesh Babu  
DecomposeMe: Simplifying ConvNets for End-to-End Learning Jose Alvarez, Lars Petersson  
Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation Seunghoon Hong, Hyeonwoo Noh, Bohyung Han  
DeepBox: Learning Objectness with Convolutional Networks Weicheng Kuo, Bharath Hariharan, Jitendra Malik  
DeepFont: Identify Your Font from An Image Zhangyang Wang, Jianchao Yang, Hailin Jin, Eli Shechtman, Aseem Agarwala, Jonathan Brandt, Thomas S. Huang  
DeepFool: a simple and accurate method to fool deep neural networks Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Pascal Frossard  
DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection Wanli Ouyang, Xiaogang Wang, Xingyu Zeng, Shi Qiu, Ping Luo, Yonglong Tian, Hongsheng Li, Shuo Yang, Zhe Wang, Chen-Change Loy, Xiaoou Tang  
DeepLogo: Hitting Logo Recognition with the Deep Neural Network Hammer Forrest N. Iandola, Anting Shen, Peter Gao, Kurt Keutzer  
DeepProposal: Hunting Objects by Cascading Deep Convolutional Layers Amir Ghodrati, Ali Diba, Marco Pedersoli, Tinne Tuytelaars, Luc Van Gool  
DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection Xi Li, Liming Zhao, Lina Wei, MingHsuan Yang, Fei Wu, Yueting Zhuang, Haibin Ling, Jingdong Wang  
Deep Attention Recurrent Q-Network Ivan Sorokin, Alexey Seleznev, Mikhail Pavlov, Aleksandr Fedorov, Anastasiia Ignateva  
Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN) Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille  
Deep Compositional Question Answering with Neural Module Networks Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein  
Deep clustering: Discriminative embeddings for segmentation and separation John R. Hershey, Zhuo Chen, Jonathan Le Roux, Shinji Watanab  
Deep CNN Ensemble with Data Augmentation for Object Detection Jian Guo, Stephen Gould  
Deep Colorization Zezhou Cheng, Qingxiong Yang, Bin Sheng  
Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data Lisa Anne Hendricks, Subhashini Venugopalan, Marcus Rohrbach, Raymond Mooney, Kate Saenko, Trevor Darrell  
Deep Convolutional Matching Jerome Revaud, Philippe Weinzaepfel, Zaid Harchaoui, Cordelia Schmid  
Deep Convolutional Networks are Hierarchical Kernel Machines Fabio Anselmi, Lorenzo Rosasco, Cheston Tan, Tomaso Poggio  
Deep Convolutional Neural Network Design Patterns Leslie N. Smith, Nicholay Topin  
Deep Directed Generative Models with Energy-Based Probability Estimation Taesup Kim, Yoshua Bengio  
Deep Convolutional Networks on Graph-Structured Data Mikael Henaff, Joan Bruna, Yann LeCun  
Deep Fishing: Gradient Features from Deep Nets Albert Gordo, Adrien Gaidon, Florent Perronnin  
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Emily Denton, Soumith Chintala, Arthur Szlam, Rob Fergus  
Deep Kernel Learning Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing  
Deep Knowledge Tracing Chris Piech, Jonathan Spencer, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas, Jascha Sohl-Dickstein  
Deep Learning Face Attributes in the Wild Ziwei Liu, Ping Luo, Xiaogang Wang, Xiaoou Tang  
Deep Learning with Differential Privacy Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, Li Zhang  
Deep Learning with S-shaped Rectified Linear Activation Units Xiaojie Jin, Chunyan Xu, Jiashi Feng, Yunchao Wei, Junjun Xiong, Shuicheng Yan  
DeepMath - Deep Sequence Models for Premise Selection Alex A. Alemi, Francois Chollet, Geoffrey Irving, Christian Szegedy, Josef Urban  
Deep multi-scale video prediction beyond mean square error Michael Mathieu, Camille Couprie, Yann LeCun  
Deep Networks Resemble Human Feed-forward Vision in Invariant Object Recognition Saeed Reza Kheradpisheh, Masoud Ghodrati, Mohammad Ganjtabesh, Timothée Masquelier  
Deep Networks with Internal Selective Attention through Feedback Connections Marijn Stollenga, Jonathan Masci, Faustino Gomez, Jürgen Schmidhuber  
Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition Radoslaw M. Cichy, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, Aude Oliva  
DeepProposals: Hunting Objects and Actions by Cascading Deep Convolutional Layers Amir Ghodrati, Ali Diba, Marco Pedersoli, Tinne Tuytelaars, Luc Van Gool  
Deep Recurrent Q-Learning for Partially Observable MDPs Matthew Hausknecht, Peter Stone  
Deep Residual Learning for Image Recognition Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun  
Deeply-Recursive Convolutional Network for Image Super-Resolution.pdf Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee  
Deep Reinforcement Learning in Parameterized Action Space Matthew Hausknecht, Peter Stone  
Deep Reinforcement Learning with an Unbounded Action Space Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, Mari Ostendorf  
Deep Reinforcement Learning with Double Q-learning Hado van Hasselt, Arthur Guez, David Silver  
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images Shuran Song, Jianxiong Xiao  
Deep SimNets Nadav Cohen, Or Sharir, Amnon Shashua  
Deep Speech: Scaling up end-to-end speech recognition Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, Andrew Y. Ng  
Deep Spiking Networks Peter O’Connor, Max Welling  
Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks Peter Ondruska, Ingmar Posner  
Deep Transfer Learning with Joint Adaptation Networks Mingsheng Long, Jianmin Wang, Michael I. Jordan  
Deep Visual-Semantic Alignments for Generating Image Descriptions Andrej Karpathy, Fei-Fei Li  
Deeply Improved Sparse Coding for Image Super-Resolution Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, Thomas Huang  
DeePM: A Deep Part-Based Model for Object Detection and Semantic Part Localization Jun Zhu, Xianjie Chen, Alan L. Yuille  
Delving Deeper into Convolutional Networks for Learning Video Representations Nicolas Ballas, Li Yao, Chris Pal, Aaron Courville  
Denoising Criterion for Variational Auto-Encoding Framework Daniel Jiwoong Im, Sungjin Ahn, Roland Memisevic, Yoshua Bengio  
DenseCap: Fully Convolutional Localization Networks for Dense Captioning Justin Johnson, Andrej Karpathy, Li Fei-Fei  
DenseBox: Unifying Landmark Localization with End to End Object Detection Lichao Huang, Yi Yang, Yafeng Deng, Yinan Yu  
Describing Multimedia Content using Attention-based Encoder–Decoder Networks Kyunghyun Cho, Aaron Courville, Yoshua Bengio  
Describing Videos by Exploiting Temporal Structure Li Yao, Atousa Torabi, Kyunghyun Cho, Nicolas Ballas, Christopher Pal, Hugo Larochelle, Aaron Courville  
Detecting Interrogative Utterances with Recurrent Neural Networks Junyoung Chung, Jacob Devlin, Hany Hassan Awadalla  
Dictionary Learning and Sparse Coding for Third-order Super-symmetric Tensors Piotr Koniusz, Anoop Cherian  
Digging Deep into the layers of CNNs: In Search of How CNNs Achieve View Invariance Amr Bakry, Mohamed Elhoseiny, Tarek El-Gaaly, Ahmed Elgammal  
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks Alexey Dosovitskiy, Philipp Fischer, Jost Tobias Springenberg, Martin Riedmiller, Thomas Brox  
DisturbLabel: Regularizing CNN on the Loss Layer Lingxi Xie, Jingdong Wang, Zhen Wei, Meng Wang, Qi Tian  
Distributed Deep Learning Using Synchronous Stochastic Gradient Descent Dipankar Das, Sasikanth Avancha, Dheevatsa Mudigere, Karthikeyan Vaidynathan, Srinivas Sridharan, Dhiraj Kalamkar, Bharat Kaul, Pradeep Dubey  
Distributed Deep Q-Learning Hao Yi Ong, Kevin Chavez, Augustus Hong  
Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition? Pooya Khorrami, Tom Le Paine, Thomas S. Huang  
Do semantic parts emerge in Convolutional Neural Networks? Abel Gonzalez-Garcia, Davide Modolo, Vittorio Ferrari  
DRAW: A Recurrent Neural Network For Image Generation Karol Gregor, Ivo Danihelka, Alex Graves, Daan Wierstra  
Drawing and Recognizing Chinese Characters with Recurrent Neural Network Xu-Yao Zhang, Fei Yin, Yan-Ming Zhang, Cheng-Lin Liu, Yoshua Bengio  
DropNeuron: Simplifying the Structure of Deep Neural Networks Wei Pan, Hao Dong, Yike Guo  
Dropout: A Simple Way to Prevent Neural Networks from Overfitting Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov  
Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho, Yoshua Bengio  
DynaNewton - Accelerating Newton’s Method for Machine Learning Hadi Daneshmand, Aurelien Lucchi, Thomas Hofmann  
EIE: Efficient Inference Engine on Compressed Deep Neural Network Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A. Horowitz, William J. Dally  
Empirical performance upper bounds for image and video captioning Li Yao, Nicolas Ballas, Kyunghyun Cho, John R. Smith, Yoshua Bengio  
End-to-End Attention-based Large Vocabulary Speech Recognition Dzmitry Bahdanau, Jan Chorowski, Dmitriy Serdyuk, Philemon Brakel, Yoshua Bengio  
End-to-End Deep Learning for Person Search Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, Xiaogang Wang  
End-to-end Learning of Action Detection from Frame Glimpses in Videos Serena Yeung, Olga Russakovsky, Greg Mori, Li Fei-Fei  
End-To-End Memory Networks Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus  
End-to-end people detection in crowded scenes Russell Stewart, Mykhaylo Andriluka  
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello  
Evaluating the visualization of what a Deep Neural Network has learned Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Bach, Klaus-Robert Müller  
Exploring the Limits of Language Modeling Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, Yonghui Wu  
FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff, Dmitry Kalenichenko, James Philbin  
Factors in Finetuning Deep Model for object detection Wanli Ouyang, Xiaogang Wang, Cong Zhang, Xiaokang Yang  
Fast Algorithms for Convolutional Neural Networks Andrew Lavin  
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) Djork-Arné Clevert, Thomas Unterthiner, Sepp Hochreiter  
Fast-Classifying, High-Accuracy Spiking Deep Networks Through Weight and Threshold Balancing Peter U. Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer  
Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition Haşim Sak, Andrew Senior, Kanishka Rao, Françoise Beaufays  
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun  
Feature-based Attention in Convolutional Neural Networks Grace W. Lindsay  
Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems Colin Raffel, Daniel P. W. Ellis  
FireCaffe: near-linear acceleration of deep neural network training on compute clusters Forrest N. Iandola, Khalid Ashraf, Mattthew W. Moskewicz, Kurt Keutzer  
First Step toward Model-Free, Anonymous Object Tracking with Recurrent Neural Networks Quan Gan, Qipeng Guo, Zheng Zhang, Kyunghyun Cho  
FitNets: Hints for Thin Deep Nets Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, Yoshua Bengio  
FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg, Philip Häusser, Caner Hazırbaş, Vladimir Golkov, Patrick van der Smagt, Daniel Cremers, Thomas Brox  
From Facial Parts Responses to Face Detection: A Deep Learning Approach Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang  
Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer, Jonathan Long, Trevor Darrell  
Fusing Multi-Stream Deep Networks for Video Classification Zuxuan Wu, Yu-Gang Jiang, Xi Wang, Hao Ye, Xiangyang Xue, Jun Wang  
Generating Images from Captions with Attention Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov  
Generating Text with Deep Reinforcement Learning Hongyu Guo  
Generative Image Modeling Using Spatial LSTMs Lucas Theis, Matthias Bethge  
Generating News Headlines with Recurrent Neural Networks Konstantin Lopyrev  
Geometry-aware Deep Transform Jiaji Huang, Qiang Qiu, Robert Calderbank, Guillermo Sapiro  
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean  
Gossip training for deep learning Michael Blot, David Picard, Matthieu Cord, Nicolas Thome  
Gradual DropIn of Layers to Train Very Deep Neural Networks Leslie N. Smith, Emily M. Hand, Timothy Doster  
Grid Long Short-Term Memory Nal Kalchbrenner, Ivo Danihelka, Alex Graves  
Guiding Long-Short Term Memory for Image Caption Generation Xu Jia, Efstratios Gavves, Basura Fernando, Tinne Tuytelaars  
Harvesting Discriminative Meta Objects with Deep CNN Features for Scene Classification Ruobing Wu, Baoyuan Wang, Wenping Wang, Yizhou Yu  
Hierarchical Attention Networks Paul Hongsuck Seo, Zhe Lin, Scott Cohen, Xiaohui Shen, Bohyung Han  
Hierarchical Object Detection with Deep Reinforcement Learning Miriam Bellver, Xavier Giro-i-Nieto, Ferran Marques, Jordi Torres  
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning Pingbo Pan, Zhongwen Xu, Yi Yang, Fei Wu, Yueting Zhuang  
Highway Networks Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber  
How far can we go without convolution: Improving fully-connected networks Zhouhan Lin, Roland Memisevic, Kishore Konda  
How Important is Weight Symmetry in Backpropagation? Qianli Liao, Joel Z. Leibo, Tomaso Poggio  
How to scale distributed deep learning? Peter H. Jin, Qiaochu Yuan, Forrest Iandola, Kurt Keutzer  
Human Action Recognition using Factorized Spatio-Temporal Convolutional Networks Lin Sun, Kui Jia, Dit-Yan Yeung, Bertram E. Shi  
Human-level concept learning through probabilistic program induction Brenden M. Lake, Ruslan Salakhutdinov, Joshua B. Tenenbau  
Human-level control through deep reinforcement learning Google DeepMind  
iCaRL: Incremental Classifier and Representation Learning Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Christoph H. Lampert  
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton  
Image Captioning with an Intermediate Attributes Layer Qi Wu, Chunhua Shen, Anton van den Hengel, Lingqiao Liu, Anthony Dick  
Image Reconstruction from Bag-of-Visual-Words Hiroharu Kato, Tatsuya Harada  
Image Representations and New Domains in Neural Image Captioning Jack Hessel, Nicolas Savva, Michael J. Wilber  
Image Super-Resolution Using Deep Convolutional Networks Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang  
Image Question Answering: A Visual Semantic Embedding Model and a New Dataset Mengye Ren, Ryan Kiros, Richard Zemel  
Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction Hyeonwoo Noh, Paul Hongsuck Seo, Bohyung Han  
Importance Weighted Autoencoders Yuri Burda, Roger Grosse, Ruslan Salakhutdinov  
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke  
Indexing of CNN Features for Large Scale Image Search Ruoyu Liu, Yao Zhao, Shikui Wei, Zhenfeng Zhu, Lixin Liao, Shuang Qiu  
InterActive: Inter-Layer Activeness Propagation Lingxi Xie, Liang Zheng, Jingdong Wang, Alan Yuille, Qi Tian  
Inverting Convolutional Networks with Convolutional Networks Alexey Dosovitskiy, Thomas Brox  
Is Image Super-resolution Helpful for Other Vision Tasks? Dengxin Dai, Yujian Wang, Yuhua Chen, Luc Van Gool  
Is L2 a Good Loss Function for Neural Networks for Image Processing? Hang Zhao, Orazio Gallo, Iuri Frosio, Jan Kautz  
Joint Calibration for Semantic Segmentation Holger Caesar, Jasper Uijlings, Vittorio Ferrari  
Large-scale Simple Question Answering with Memory Networks Antoine Bordes, Nicolas Usunier, Sumit Chopra, Jason Weston  
Large Margin Deep Neural Networks: Theory and Algorithms Shizhao Sun, Wei Chen, Liwei Wang, Tie-Yan Liu  
Layer Normalization Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton  
Learning Contextual Dependencies with Convolutional Hierarchical Recurrent Neural Networks Zhen Zuo, Bing Shuai, Gang Wang, Xiao Liu, Xingxing Wang, Bing Wang  
Learning Deconvolution Network for Semantic Segmentation Hyeonwoo Noh, Seunghoon Hong, Bohyung Han  
Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification Tong Xiao, Hongsheng Li, Wanli Ouyang, Xiaogang Wang  
Learning Deep Representations of Fine-grained Visual Descriptions Scott Reed, Zeynep Akata, Bernt Schiele, Honglak Lee  
Learning Fine-grained Features via a CNN Tree for Large-scale Classification Zhenhua Wang, Xingxing Wang, Gang Wang  
Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille  
Learning from LDA using Deep Neural Networks Dongxu Zhang, Tianyi Luo, Dong Wang, Rong Liu  
Learning Multiple Tasks with Deep Relationship Networks Mingsheng Long, Jianmin Wang  
Learning scale-variant and scale-invariant features for deep image classification Nanne van Noord, Eric Postma  
Learning Spatiotemporal Features with 3D Convolutional Networks Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, Manohar Paluri  
Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, Shimon Whiteson  
Learning to Compose Neural Networks for Question Answering Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein  
Learning to learn by gradient descent by gradient descent Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Nando de Freitas  
Learning to Linearize Under Uncertainty Ross Goroshin, Michael Mathieu, Yann LeCun  
Learning to reinforcement learn Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, Matt Botvinick  
Learning to See by Moving Pulkit Agrawal, Joao Carreira, Jitendra Malik  
Learning to Segment Object Candidates Pedro O. Pinheiro, Ronan Collobert, Piotr Dollar  
Learning to Track at 100 FPS with Deep Regression Networks David Held, Sebastian Thrun, Silvio Savarese  
Learning to track for spatio-temporal action localization Philippe Weinzaepfel, Zaid Harchaoui, Cordelia Schmid  
Learning Transferable Features with Deep Adaptation Networks Mingsheng Long, Yue Cao, Jianmin Wang, Michael I. Jordan  
Learning Wake-Sleep Recurrent Attention Models Jimmy Ba, Roger Grosse, Ruslan Salakhutdinov, Brendan Frey  
Learning Visual Features from Large Weakly Supervised Data Armand Joulin, Laurens van der Maaten, Allan Jabri, Nicolas Vasilache  
Leveraging Context to Support Automated Food Recognition in Restaurants Vinay Bettadapura, Edison Thomaz, Aman Parnami, Gregory Abowd, Irfan Essa  
Lipreading with Long Short-Term Memory Michael Wand, Jan Koutník, Jürgen Schmidhuber  
Listen, Attend and Spell William Chan, Navdeep Jaitly, Quoc V. Le, Oriol Vinyals  
Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences Hongyuan Mei, Mohit Bansal, Matthew R. Walter  
LLNet: A Deep Autoencoder Approach to Natural Low-light Image Enhancement Kin Gwn Lore, Adedotun Akintayo, Soumik Sarkar  
Locally-Supervised Deep Hybrid Model for Scene Recognition Sheng Guo, Weilin Huang, Yu Qiao  
LocNet: Improving Localization Accuracy for Object Detection Spyros Gidaris, Nikos Komodakis  
LOGO-Net: Large-scale Deep Logo Detection and Brand Recognition with Deep Region-based Convolutional Networks Steven C.H. Hoi, Xiongwei Wu, Hantang Liu, Yue Wu, Huiqiong Wang, Hui Xue, Qiang Wu  
Long Short-Term Memory-Networks for Machine Reading Jianpeng Cheng, Li Dong, Mirella Lapata  
Long-term Recurrent Convolutional Networks for Visual Recognition and Description Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, Trevor Darrell  
Love Thy Neighbors: Image Annotation by Exploiting Image Metadata Justin Johnson, Lamberto Ballan, Fei-Fei Li  
MADE: Masked Autoencoder for Distribution Estimation Mathieu Germain, Karol Gregor, Iain Murray, Hugo Larochelle  
Manitest: Are classifiers really invariant? Alhussein Fawzi, Pascal Frossard  
Maxout Networks Ian J. Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, Yoshua Bengio  
Memory-Efficient Backpropagation Through Time Audrūnas Gruslys, Remi Munos, Ivo Danihelka, Marc Lanctot, Alex Graves  
Modelling Uncertainty in Deep Learning for Camera Relocalization Alex Kendall, Roberto Cipolla  
MuFuRU: The Multi-Function Recurrent Unit Dirk Weissenborn, Tim Rocktäschel  
Multiagent Cooperation and Competition with Deep Reinforcement Learning Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, Raul Vicente  
Multi-Instance Visual-Semantic Embedding Zhou Ren, Hailin Jin, Zhe Lin, Chen Fang, Alan Yuille  
Multi-task Sequence to Sequence Learning Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, Lukasz Kaiser  
Multi-view Machines Bokai Cao, Hucheng Zhou, Philip S. Yu  
Multimodal Deep Learning for Robust RGB-D Object Recognition Andreas Eitel, Jost Tobias Springenberg, Luciano Spinello, Martin Riedmiller, Wolfram Burgard  
MuProp: Unbiased Backpropagation for Stochastic Neural Networks Shixiang Gu, Sergey Levine, Ilya Sutskever, Andriy Mnih  
Named Entity Recognition with Bidirectional LSTM-CNNs Jason P.C. Chiu, Eric Nichols  
Natural Neural Networks Guillaume Desjardins, Karen Simonyan, Razvan Pascanu, Koray Kavukcuoglu  
Neighborhood Watch: Stochastic Gradient Descent with Neighbors Thomas Hofmann, Aurelien Lucchi, Brian McWilliams  
Net2Net: Accelerating Learning via Knowledge Transfer Tianqi Chen, Ian Goodfellow, Jonathon Shlens  
Neural Combinatorial Optimization with Reinforcement Learning Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, Samy Bengio  
Neural Functional Programming John K. Feser, Marc Brockschmidt, Alexander L. Gaunt, Daniel Tarlow  
Neural GPUs Learn Algorithms Łukasz Kaiser, Ilya Sutskever  
Neural Random-Access Machines Karol Kurach, Marcin Andrychowicz, Ilya Sutskever  
Neural Semantic Encoders Tsendsuren Munkhdalai, Hong Yu  
Object Recognition with and without Objects Zhuotun Zhu, Lingxi Xie, Alan L. Yuille  
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models Juergen Schmidhuber  
On Multiplicative Integration with Recurrent Neural Networks Yuhuai Wu, Saizheng Zhang, Ying Zhang, Yoshua Bengio, Ruslan Salakhutdinov  
On the Convergence of SGD Training of Neural Networks Thomas M. Breuel  
On the Expressive Power of Deep Learning: A Tensor Analysis Nadav Cohen, Or Sharir, Amnon Shashua  
On the expressive power of deep neural networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein  
One-Shot Video Object Segmentation Sergi Caelles, Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Laura Leal-Taixé, Daniel Cremers, Luc Van Gool  
Online and Offline Handwritten Chinese Character Recognition: A Comprehensive Study and New Benchmark Xu-Yao Zhang, Yoshua Bengio, Cheng-Lin Liu  
Online Batch Selection for Faster Training of Neural Networks Ilya Loshchilov, Frank Hutter  
Orthogonal RNNs and Long-Memory Tasks Mikael Henaff, Arthur Szlam, Yann LeCun  
Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation Marijn F. Stollenga, Wonmin Byeon, Marcus Liwicki, Juergen Schmidhuber  
Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations Behnam Neyshabur, Yuhuai Wu, Ruslan Salakhutdinov, Nathan Srebro  
Path-SGD: Path-Normalized Optimization in Deep Neural Networks Behnam Neyshabur, Ruslan Salakhutdinov, Nathan Srebro  
Person Recognition in Personal Photo Collections Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele  
Pixel Recurrent Neural Networks Aaron van den Oord, Nal Kalchbrenner, Koray Kavukcuoglu  
PlaNet - Photo Geolocation with Convolutional Neural Networks Tobias Weyand, Ilya Kostrikov, James Philbin  
Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Nikolai Yakovenko, Liangliang Cao, Colin Raffel, James Fan  
Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions Jimmy Ba, Kevin Swersky, Sanja Fidler, Ruslan Salakhutdinov  
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks José Miguel Hernández-Lobato, Ryan P. Adams  
ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networks Chen Sun, Manohar Paluri, Ronan Collobert, Ram Nevatia, Lubomir Bourde  
Proposal-free Network for Instance-level Object Segmentation Xiaodan Liang, Yunchao Wei, Xiaohui Shen, Jianchao Yang, Liang Lin, Shuicheng Yan  
P-CNN: Pose-based CNN Features for Action Recognition Guilhem Chéron, Ivan Laptev, Cordelia Schmid  
R-CNN minus R Karel Lenc, Andrea Vedaldi  
RAID: A Relation-Augmented Image Descriptor Paul Guerrero, Niloy J. Mitra, Peter Wonka  
RATM: Recurrent Attentive Tracking Model Samira Ebrahimi Kahou, Vincent Michalski, Roland Memisevic  
Random Maxout Features Youssef Mroueh, Steven Rennie, Vaibhava Goel  
Recurrent Attention Models for Depth-Based Person Identification Albert Haque, Alexandre Alahi, Li Fei-Fei  
Recurrent Attentional Networks for Saliency Detection Jason Kuen, Zhenhua Wang, Gang Wang  
Recurrent Batch Normalization Tim Cooijmans, Nicolas Ballas, César Laurent, Aaron Courvill  
Recurrent Instance Segmentation Bernardino Romera-Paredes, Philip H. S. Torr  
Recurrent Models of Visual Attention Volodymyr Mnih, Nicolas Heess, Alex Graves, Koray Kavukcuoglu  
Recurrent Network Models for Kinematic Tracking Katerina Fragkiadaki, Sergey Levine, Jitendra Malik  
Recurrent Neural Networks for Driver Activity Anticipation via Sensory-Fusion Architecture Ashesh Jain, Avi Singh, Hema S Koppula, Shane Soh, Ashutosh Saxena  
Recurrent Neural Networks With Limited Numerical Precision Joachim Ott, Zhouhan Lin, Ying Zhang, Shih-Chii Liu, Yoshua Bengio  
Recurrent Reinforcement Learning: A Hybrid Approach Xiujun Li, Lihong Li, Jianfeng Gao, Xiaodong He, Jianshu Chen, Li Deng, Ji He  
Recursive Decomposition for Nonconvex Optimization Abram L. Friesen, Pedro Domingos  
Recursive Training of 2D-3D Convolutional Networks for Neuronal Boundary Detection Kisuk Lee, Aleksandar Zlateski, Ashwin Vishwanathan, H. Sebastian Seung  
Relative Natural Gradient for Learning Large Complex Models Ke Sun, Frank Nielsen  
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views Hao Su, Charles R. Qi, Yangyan Li, Leonidas Guibas  
ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks Francesco Visin, Kyle Kastner, Kyunghyun Cho, Matteo Matteucci, Aaron Courville, Yoshua Bengio  
ReSeg: A Recurrent Neural Network for Object Segmentation Francesco Visin, Kyle Kastner, Aaron Courville, Yoshua Bengio, Matteo Matteucci, Kyunghyun Cho  
Reuse of Neural Modules for General Video Game Playing Alexander Braylan, Mark Hollenbeck, Elliot Meyerson, Risto Miikkulainen  
Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik  
RL2: Fast Reinforcement Learning via Slow Reinforcement Learning Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel  
Scaling Up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix Roger B. Grosse, Ruslan Salakhutdinov  
SceneNet: Understanding Real World Indoor Scenes With Synthetic Data Ankur Handa, Viorica Patraucean, Vijay Badrinarayanan, Simon Stent, Roberto Cipolla  
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer  
Search-Convolutional Neural Networks James Atwood, Don Towsley  
Searching for Higgs Boson Decay Modes with Deep Learning Peter Sadowski, Pierre Baldi, Daniel Whiteson  
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla  
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling Vijay Badrinarayanan, Ankur Handa, Roberto Cipolla  
Semantic Image Segmentation via Deep Parsing Network Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, Xiaoou Tang  
Semi-supervised Sequence Learning Andrew M. Dai, Quoc V. Le  
SentiCap: Generating Image Descriptions with Sentiments Alexander Mathews, Lexing Xie, Xuming He  
Singularity of the Hessian in Deep Learning Levent Sagun, Leon Bottou, Yann LeCun  
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size Forrest N. Iandola, Matthew W. Moskewicz, Khalid Ashraf, Song Han, William J. Dally, Kurt Keutzer  
Show and Tell: A Neural Image Caption Generator Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erha  
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio  
Simple Baseline for Visual Question Answering Bolei Zhou, Yuandong Tian, Sainbayar Sukhbaatar, Arthur Szlam, Rob Fergus  
Skip-Thought Vectors Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, Sanja Fidler  
Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks Arash Ardakani, Carlo Condo, Warren J. Gross  
Sparsifying Neural Network Connections for Face Recognition Yi Sun, Xiaogang Wang, Xiaoou Tang  
Spatial Semantic Regularisation for Large Scale Object Detection Damian Mrowca, Marcus Rohrbach, Judy Hoffman, Ronghang Hu, Kate Saenko, Trevor Darrell  
Spatial Transformer Networks Max Jaderberg, Karen Simonyan, Andrew Zisserman, Koray Kavukcuoglu  
Spiking Deep Networks with LIF Neurons Eric Hunsberger, Chris Eliasmith  
Stacked Attention Networks for Image Question Answering Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola  
Stacked What-Where Auto-encoders Junbo Zhao, Michael Mathieu, Ross Goroshin, Yann Lecun  
STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation Yunchao Wei, Xiaodan Liang, Yunpeng Chen, Xiaohui Shen, Ming-Ming Cheng, Yao Zhao, Shuicheng Yan  
STDP-based spiking deep neural networks for object recognition Saeed Reza Kheradpisheh, Mohammad Ganjtabesh, Simon J Thorpe, Timothée Masquelier  
Stochastic Gradient Made Stable: A Manifold Propagation Approach for Large-Scale Optimization Yadong Mu, Wei Liu, Wei Fan  
StochasticNet: Forming Deep Neural Networks via Stochastic Connectivity Mohammad Javad Shafiee, Parthipan Siva, Alexander Wong  
Strategic Attentive Writer for Learning Macro-Actions Alexander (Sasha) Vezhnevets, Volodymyr Mnih, John Agapiou, Simon Osindero, Alex Graves, Oriol Vinyals, Koray Kavukcuoglu  
Studying Very Low Resolution Recognition Using Deep Networks Zhangyang Wang, Shiyu Chang, Yingzhen Yang, Ding Liu, Thomas S. Huang  
Super-Resolution with Deep Convolutional Sufficient Statistics Joan Bruna, Pablo Sprechmann, Yann LeCun  
Superpixel Convolutional Networks using Bilateral Inceptions Raghudeep Gadde, Varun Jampani, Martin Kiefel, Peter V. Gehler  
Structured Depth Prediction in Challenging Monocular Video Sequences Miaomiao Liu, Mathieu Salzmann, Xuming He  
Structured Memory for Neural Turing Machines Wei Zhang, Yang Yu  
Symmetry-invariant optimization in deep networks Vijay Badrinarayanan, Bamdev Mishra, Roberto Cipolla  
Tagger: Deep Unsupervised Perceptual Grouping Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hotloo Hao, Jürgen Schmidhuber, Harri Valpola  
Task Loss Estimation for Sequence Prediction Dzmitry Bahdanau, Dmitriy Serdyuk, Philémon Brakel, Nan Rosemary Ke, Jan Chorowski, Aaron Courville, Yoshua Bengio  
T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos Kai Kang, Hongsheng Li, Junjie Yan, Xingyu Zeng, Bin Yang, Tong Xiao, Cong Zhang, Zhe Wang, Ruohui Wang, Xiaogang Wang, Wanli Ouyang  
Teaching Machines to Read and Comprehend Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, Phil Blunsom  
Text-Attentional Convolutional Neural Networks for Scene Text Detection Tong He, Weilin Huang, Yu Qiao, Jian Yao  
The Effects of Hyperparameters on SGD Training of Neural Networks Thomas M. Breuel  
The Unreasonable Effectiveness of Recurrent Neural Networks Andrej Karpathy  
Towards Automatic Image Editing: Learning to See another You Amir Ghodrati, Xu Jia, Marco Pedersoli, Tinne Tuytelaars  
Towards Biologically Plausible Deep Learning Yoshua Bengio, Dong-Hyun Lee, Jorg Bornschein, Zhouhan Lin  
Towards Good Practices for Very Deep Two-Stream ConvNets Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao  
Towards universal neural nets: Gibbs machines and ACE Galin Georgiev  
Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control Fangyi Zhang, Juergen Leitner, Michael Milford, Ben Upcroft, Peter Corke  
Train faster, generalize better: Stability of stochastic gradient descent Moritz Hardt, Benjamin Recht, Yoram Singer  
Training a Convolutional Neural Network for Appearance-Invariant Place Recognition Ruben Gomez-Ojeda, Manuel Lopez-Antequera, Nicolai Petkov, Javier Gonzalez-Jimenez  
Training Deep Networks with Structured Layers by Matrix Backpropagation Catalin Ionescu, Orestis Vantzos, Cristian Sminchisescu  
Training Deeper Convolutional Networks with Deep Supervision Liwei Wang, Chen-Yu Lee, Zhuowen Tu, Svetlana Lazebnik  
Trainable performance upper bounds for image and video captioning Li Yao, Nicolas Ballas, Kyunghyun Cho, John R. Smith, Yoshua Bengio  
Training Very Deep Networks Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber  
Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping Michael Xie, Neal Jean, Marshall Burke, David Lobell, Stefano Ermon  
Translating Videos to Natural Language Using Deep Recurrent Neural Networks Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, Kate Saenko  
Unconstrained Face Verification using Deep CNN Features Jun-Cheng Chen, Vishal M. Patel, Rama Chellappa  
Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization Roy Frostig, Rong Ge, Sham M. Kakade, Aaron Sidford  
Understanding deep learning requires rethinking generalization Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, Oriol Vinyals  
Understanding Locally Competitive Networks Rupesh Kumar Srivastava, Jonathan Masci, Faustino Gomez, Jürgen Schmidhuber  
Understanding Neural Networks Through Deep Visualization Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, Hod Lipson  
Understand Scene Categories by Objects: A Semantic Regularized Scene Classifier Using Convolutional Neural Networks Yiyi Liao, Sarath Kodagoda, Yue Wang, Lei Shi, Yong Liu  
Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders Huan Yang, Baoyuan Wang, Stephen Lin, David Wipf, Minyi Guo, Baining Guo  
Unsupervised Learning of Video Representations using LSTMs Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov  
Unsupervised Learning of Visual Representations using Videos Xiaolong Wang, Abhinav Gupta  
Unsupervised Semantic Parsing of Video Collections Ozan Sener, Amir Zamir, Silvio Savarese, Ashutosh Saxena  
Unsupervised Visual Representation Learning by Context Prediction Carl Doersch, Abhinav Gupta, Alexei A. Efros  
Using Descriptive Video Services to Create a Large Data Source for Video Annotation Research Atousa Torabi, Christopher Pal, Hugo Larochelle, Aaron Courville  
Using Fast Weights to Attend to the Recent Past Jimmy Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, Catalin Ionescu  
Variable Rate Image Compression with Recurrent Neural Networks George Toderici, Sean M. O’Malley, Sung Jin Hwang, Damien Vincent, David Minnen, Shumeet Baluja, Michele Covell, Rahul Sukthankar  
Video Paragraph Captioning using Hierarchical Recurrent Neural Networks Haonan Yu, Jiang Wang, Zhiheng Huang, Yi Yang, Wei Xu  
VISALOGY: Answering Visual Analogy Questions Fereshteh Sadeghi, C. Lawrence Zitnick, Ali Farhadi  
Visualizing and Understanding Deep Texture Representations Tsung-Yu Lin, Subhransu Maji  
Visualizing and Understanding Recurrent Networks Andrej Karpathy, Justin Johnson, Fei-Fei Li  
Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images Aravindh Mahendran, Andrea Vedaldi  
Visual7W: Grounded Question Answering in Images Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei  
Watch and Learn: Semi-Supervised Learning of Object Detectors from Videos Ishan Misra, Abhinav Shrivastava, Martial Hebert  
We Are Humor Beings: Understanding and Predicting Visual Humor Arjun Chandrasekaran, Ashwin K Vijayakumar, Stanislaw Antol, Mohit Bansal, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh  
Weakly-Supervised Alignment of Video With Text P. Bojanowski, R. Lagugie, Edouard Grave, Francis Bach, I. Laptev, J. Ponce, C. Schmid  
Weakly Supervised Cascaded Convolutional Networks Ali Diba, Vivek Sharma, Ali Pazandeh, Hamed Pirsiavash, Luc Van Gool  
Weakly Supervised Deep Detection Networks Hakan Bilen, Andrea Vedaldi  
Weight Uncertainty in Neural Networks Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, Daan Wierstra  
What is Holding Back Convnets for Detection? Bojan Pepik, Rodrigo Benenson, Tobias Ritschel, Bernt Schiele  
What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment Hongyuan Mei, Mohit Bansal, Matthew R. Walter  
What can we learn about CNNs from a large scale controlled object dataset? Ali Borji, Saeed Izadi, Laurent Itti  
Where To Look: Focus Regions for Visual Question Answering Kevin J. Shih, Saurabh Singh, Derek Hoiem  
Who’s Behind the Camera? Identifying the Authorship of a Photograph Christopher Thomas, Adriana Kovashka  
Why Regularized Auto-Encoders learn Sparse Representation? Devansh Arpit, Yingbo Zhou, Hung Ngo, Venu Govindaraju  
Wide Residual Networks Sergey Zagoruyko, Nikos Komodakis  
WordRank: Learning Word Embeddings via Robust Ranking Shihao Ji, Hyokun Yun, Pinar Yanardag, Shin Matsushima, S. V. N. Vishwanathan  
YodaNN: An Ultra-Low Power Convolutional Neural Network Accelerator Based on Binary Weights Renzo Andri, Lukas Cavigelli, Davide Rossi, Luca Benini  
You Only Look Once: Unified, Real-Time Object Detection Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi  
Zero-Shot Learning via Semantic Similarity Embedding Ziming Zhang, Venkatesh Saligrama  
ZNN - A Fast and Scalable Algorithm for Training 3D Convolutional Networks on Multi-Core and Many-Core Shared Memory Machines Aleksandar Zlateski, Kisuk Lee, H. Sebastian Seung  
Zoom Better to See Clearer: Huamn Part Segmentation with Auto Zoom Net Fangting Xia, Peng Wang, Liang-Chieh Chen, Alan L. Yuille  

Dataset

Title Author or Source Tags
Caltech 101 L. Fei-Fei, R. Fergus and P. Perona  
Caltech 256 G. Griffin, AD. Holub, P. Perona  
CIFAR-10 Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton  
CIFAR-100 Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton  
The Comprehensive Cars (CompCars) dataset Linjie Yang, Ping Luo, Chen Change Loy, Xiaoou Tang  
Flickr30k Peter Young, Alice Lai, Micah Hodosh, Julia Hockenmaier  
ImageNet Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg and Li Fei-Fei  
Microsoft COCO Microsoft Research  
MNIST Yann LeCun, Corinna Cortes, Christopher J.C. Burges  
Places MIT Computer Science and Artificial Intelligence Laboratory  
STL-10 Adam Coates, Honglak Lee, Andrew Y. Ng  
SVHN Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y. Ng  
TGIF Yuncheng Li, Yale Song, Liangliang Cao, Joel Tetreault, Larry Goldberg, Alejandro Jaimes, Jiebo Luo  
Visual Perception of Forest Trails IDSIA, USI/SUPSI and Robotics and Perception Group, UZH  
WWW Crowd Dataset Jing Shao, Kai Kang, Chen Change Loy, and Xiaogang Wang  
Open Images Dataset Google Inc  
YouTube-BoundingBoxes Dataset Google Research  

Podcast, Talks, etc.

Title Author or Source Tags
Talking Machines hosted by Katherine Gorman and Ryan Adams  
Machine Learning & Computer Vision Talks computervisiontalks  
How we’re teaching computers to understand pictures Fei-Fei Li, Stanford University  
Deep Learning Community    

Codes (not maintain anymore, still useful for beginners)

This piece of code was written when I was preparing my Deep Learning workshop. For beginner, I believe that this show a simple enough example for building Deep Neural Networks.Although I stopped developing this package, but you can still ask questions about it through issues, I will reply as soon as possible.

  • Telauges
    • A new deep learning library for learning DL.
    • MLP Layers: Tanh Layer, Sigmoid Layer, Identity Layer, ReLU Layer
    • Softmax Regression
    • ConvNet layers: Tanh Layer, Sigmoid Layer, Identity Layer, ReLU Layer
    • Max-Pooling layer
    • Max-Pooling same size
    • Feedforward Model
    • Auto-Encoder Model
    • SGD, Adagrad, Adadelta, RMSprop, Adam
    • Dropout