awesome-awesomeness/html/deeplearningpapers.html

<h1 id="awesome---most-cited-deep-learning-papers">Awesome - Most Cited
Deep Learning Papers</h1>
<p><a href="https://github.com/sindresorhus/awesome"><img
src="https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg"
alt="Awesome" /></a></p>
<p>[Notice] This list is not being maintained anymore because of the
overwhelming amount of deep learning papers published every day since
2017.</p>
<p>A curated list of the most cited deep learning papers (2012-2016)</p>
<p>We believe that there exist <em>classic</em> deep learning papers
which are worth reading regardless of their application domain. Rather
than providing overwhelming amount of papers, We would like to provide a
<em>curated list</em> of the awesome deep learning papers which are
considered as <em>must-reads</em> in certain research domains.</p>
<h2 id="background">Background</h2>
<p>Before this list, there exist other <em>awesome deep learning
lists</em>, for example, <a
href="https://github.com/kjw0612/awesome-deep-vision">Deep Vision</a>
and <a href="https://github.com/kjw0612/awesome-rnn">Awesome Recurrent
Neural Networks</a>. Also, after this list comes out, another awesome
list for deep learning beginners, called <a
href="https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap">Deep
Learning Papers Reading Roadmap</a>, has been created and loved by many
deep learning researchers.</p>
<p>Although the <em>Roadmap List</em> includes lots of important deep
learning papers, it feels overwhelming for me to read them all. As I
mentioned in the introduction, I believe that seminal works can give us
lessons regardless of their application domain. Thus, I would like to
introduce <strong>top 100 deep learning papers</strong> here as a good
starting point of overviewing deep learning researches.</p>
<p>To get the news for newly released papers everyday, follow my <a
href="https://twitter.com/TerryUm_ML">twitter</a> or <a
href="https://www.facebook.com/terryum.io/">facebook page</a>!</p>
<h2 id="awesome-list-criteria">Awesome list criteria</h2>
<ol type="1">
<li>A list of <strong>top 100 deep learning papers</strong> published
from 2012 to 2016 is suggested.</li>
<li>If a paper is added to the list, another paper (usually from *More
Papers from 2016” section) should be removed to keep top 100 papers.
(Thus, removing papers is also important contributions as well as adding
papers)</li>
<li>Papers that are important, but failed to be included in the list,
will be listed in <em>More than Top 100</em> section.</li>
<li>Please refer to <em>New Papers</em> and <em>Old Papers</em> sections
for the papers published in recent 6 months or before 2012.</li>
</ol>
<p><em>(Citation criteria)</em> - <strong>&lt; 6 months</strong> :
<em>New Papers</em> (by discussion) - <strong>2016</strong> : +60
citations or “More Papers from 2016” - <strong>2015</strong> : +200
citations - <strong>2014</strong> : +400 citations -
<strong>2013</strong> : +600 citations - <strong>2012</strong> : +800
citations - <strong>~2012</strong> : <em>Old Papers</em> (by
discussion)</p>
<p>Please note that we prefer seminal deep learning papers that can be
applied to various researches rather than application papers. For that
reason, some papers that meet the criteria may not be accepted while
others can be. It depends on the impact of the paper, applicability to
other researches scarcity of the research domain, and so on.</p>
<p><strong>We need your contributions!</strong></p>
<p>If you have any suggestions (missing papers, new papers, key
researchers or typos), please feel free to edit and pull a request.
(Please read the <a
href="https://github.com/terryum/awesome-deep-learning-papers/blob/master/Contributing.md">contributing
guide</a> for further instructions, though just letting me know the
title of papers can also be a big contribution to us.)</p>
<p>(Update) You can download all top-100 papers with <a
href="https://github.com/terryum/awesome-deep-learning-papers/blob/master/fetch_papers.py">this</a>
and collect all authors’ names with <a
href="https://github.com/terryum/awesome-deep-learning-papers/blob/master/get_authors.py">this</a>.
Also, <a
href="https://github.com/terryum/awesome-deep-learning-papers/blob/master/top100papers.bib">bib
file</a> for all top-100 papers are available. Thanks, doodhwala, <a
href="https://github.com/sunshinemyson">Sven</a> and <a
href="https://github.com/grepinsight">grepinsight</a>!</p>
<ul>
<li>Can anyone contribute the code for obtaining the statistics of the
authors of Top-100 papers?</li>
</ul>
<h2 id="contents">Contents</h2>
<ul>
<li><a href="#understanding--generalization--transfer">Understanding /
Generalization / Transfer</a></li>
<li><a href="#optimization--training-techniques">Optimization / Training
Techniques</a></li>
<li><a href="#unsupervised--generative-models">Unsupervised / Generative
Models</a></li>
<li><a href="#convolutional-neural-network-models">Convolutional Network
Models</a></li>
<li><a href="#image-segmentation--object-detection">Image Segmentation /
Object Detection</a></li>
<li><a href="#image--video--etc">Image / Video / Etc</a></li>
<li><a href="#natural-language-processing--rnns">Natural Language
Processing / RNNs</a></li>
<li><a href="#speech--other-domain">Speech / Other Domain</a></li>
<li><a href="#reinforcement-learning--robotics">Reinforcement Learning /
Robotics</a></li>
<li><a href="#more-papers-from-2016">More Papers from 2016</a></li>
</ul>
<p><em>(More than Top 100)</em></p>
<ul>
<li><a href="#new-papers">New Papers</a> : Less than 6 months</li>
<li><a href="#old-papers">Old Papers</a> : Before 2012</li>
<li><a href="#hw--sw--dataset">HW / SW / Dataset</a> : Technical
reports</li>
<li><a href="#book--survey--review">Book / Survey / Review</a></li>
<li><a href="#video-lectures--tutorials--blogs">Video Lectures /
Tutorials / Blogs</a></li>
<li><a href="#appendix-more-than-top-100">Appendix: More than Top
100</a> : More papers not in the list</li>
</ul>
<hr />
<h3 id="understanding-generalization-transfer">Understanding /
Generalization / Transfer</h3>
<ul>
<li><strong>Distilling the knowledge in a neural network</strong>
(2015), G. Hinton et al. <a
href="http://arxiv.org/pdf/1503.02531">[pdf]</a></li>
<li><strong>Deep neural networks are easily fooled: High confidence
predictions for unrecognizable images</strong> (2015), A. Nguyen et
al. <a href="http://arxiv.org/pdf/1412.1897">[pdf]</a></li>
<li><strong>How transferable are features in deep neural
networks?</strong> (2014), J. Yosinski et al. <a
href="http://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf">[pdf]</a></li>
<li><strong>CNN features off-the-Shelf: An astounding baseline for
recognition</strong> (2014), A. Razavian et al. <a
href="http://www.cv-foundation.org//openaccess/content_cvpr_workshops_2014/W15/papers/Razavian_CNN_Features_Off-the-Shelf_2014_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>Learning and transferring mid-Level image representations
using convolutional neural networks</strong> (2014), M. Oquab et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Oquab_Learning_and_Transferring_2014_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>Visualizing and understanding convolutional
networks</strong> (2014), M. Zeiler and R. Fergus <a
href="http://arxiv.org/pdf/1311.2901">[pdf]</a></li>
<li><strong>Decaf: A deep convolutional activation feature for generic
visual recognition</strong> (2014), J. Donahue et al. <a
href="http://arxiv.org/pdf/1310.1531">[pdf]</a></li>
</ul>
<!---[Key researchers]  [Geoffrey Hinton](https://scholar.google.ca/citations?user=JicYPdAAAAAJ), [Yoshua Bengio](https://scholar.google.ca/citations?user=kukA0LcAAAAJ), [Jason Yosinski](https://scholar.google.ca/citations?hl=en&user=gxL1qj8AAAAJ) -->
<h3 id="optimization-training-techniques">Optimization / Training
Techniques</h3>
<ul>
<li><strong>Training very deep networks</strong> (2015), R. Srivastava
et al. <a
href="http://papers.nips.cc/paper/5850-training-very-deep-networks.pdf">[pdf]</a></li>
<li><strong>Batch normalization: Accelerating deep network training by
reducing internal covariate shift</strong> (2015), S. Loffe and C.
Szegedy <a href="http://arxiv.org/pdf/1502.03167">[pdf]</a></li>
<li><strong>Delving deep into rectifiers: Surpassing human-level
performance on imagenet classification</strong> (2015), K. He et al. <a
href="http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/He_Delving_Deep_into_ICCV_2015_paper.pdf">[pdf]</a></li>
<li><strong>Dropout: A simple way to prevent neural networks from
overfitting</strong> (2014), N. Srivastava et al. <a
href="http://jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf">[pdf]</a></li>
<li><strong>Adam: A method for stochastic optimization</strong> (2014),
D. Kingma and J. Ba <a
href="http://arxiv.org/pdf/1412.6980">[pdf]</a></li>
<li><strong>Improving neural networks by preventing co-adaptation of
feature detectors</strong> (2012), G. Hinton et al. <a
href="http://arxiv.org/pdf/1207.0580.pdf">[pdf]</a></li>
<li><strong>Random search for hyper-parameter optimization</strong>
(2012) J. Bergstra and Y. Bengio <a
href="http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a">[pdf]</a></li>
</ul>
<!---[Key researchers] [Geoffrey Hinton](https://scholar.google.ca/citations?user=JicYPdAAAAAJ), [Yoshua Bengio](https://scholar.google.ca/citations?user=kukA0LcAAAAJ), [Christian Szegedy](https://scholar.google.ca/citations?hl=en&user=3QeF7mAAAAAJ), [Sergey Ioffe](https://scholar.google.ca/citations?user=S5zOyIkAAAAJ), [Kaming He](https://scholar.google.ca/citations?hl=en&user=DhtAFkwAAAAJ), [Diederik P. Kingma](https://scholar.google.ca/citations?hl=en&user=yyIoQu4AAAAJ)-->
<h3 id="unsupervised-generative-models">Unsupervised / Generative
Models</h3>
<ul>
<li><strong>Pixel recurrent neural networks</strong> (2016), A. Oord et
al. <a href="http://arxiv.org/pdf/1601.06759v2.pdf">[pdf]</a></li>
<li><strong>Improved techniques for training GANs</strong> (2016), T.
Salimans et al. <a
href="http://papers.nips.cc/paper/6125-improved-techniques-for-training-gans.pdf">[pdf]</a></li>
<li><strong>Unsupervised representation learning with deep convolutional
generative adversarial networks</strong> (2015), A. Radford et al. <a
href="https://arxiv.org/pdf/1511.06434v2">[pdf]</a></li>
<li><strong>DRAW: A recurrent neural network for image
generation</strong> (2015), K. Gregor et al. <a
href="http://arxiv.org/pdf/1502.04623">[pdf]</a></li>
<li><strong>Generative adversarial nets</strong> (2014), I. Goodfellow
et al. <a
href="http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf">[pdf]</a></li>
<li><strong>Auto-encoding variational Bayes</strong> (2013), D. Kingma
and M. Welling <a href="http://arxiv.org/pdf/1312.6114">[pdf]</a></li>
<li><strong>Building high-level features using large scale unsupervised
learning</strong> (2013), Q. Le et al. <a
href="http://arxiv.org/pdf/1112.6209">[pdf]</a></li>
</ul>
<!---[Key researchers] [Yoshua Bengio](https://scholar.google.ca/citations?user=kukA0LcAAAAJ), [Ian Goodfellow](https://scholar.google.ca/citations?user=iYN86KEAAAAJ), [Alex Graves](https://scholar.google.ca/citations?user=DaFHynwAAAAJ)-->
<h3 id="convolutional-neural-network-models">Convolutional Neural
Network Models</h3>
<ul>
<li><strong>Rethinking the inception architecture for computer
vision</strong> (2016), C. Szegedy et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Szegedy_Rethinking_the_Inception_CVPR_2016_paper.pdf">[pdf]</a></li>
<li><strong>Inception-v4, inception-resnet and the impact of residual
connections on learning</strong> (2016), C. Szegedy et al. <a
href="http://arxiv.org/pdf/1602.07261">[pdf]</a></li>
<li><strong>Identity Mappings in Deep Residual Networks</strong> (2016),
K. He et al. <a
href="https://arxiv.org/pdf/1603.05027v2.pdf">[pdf]</a></li>
<li><strong>Deep residual learning for image recognition</strong>
(2016), K. He et al. <a
href="http://arxiv.org/pdf/1512.03385">[pdf]</a></li>
<li><strong>Spatial transformer network</strong> (2015), M. Jaderberg et
al., <a
href="http://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf">[pdf]</a></li>
<li><strong>Going deeper with convolutions</strong> (2015), C. Szegedy
et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Szegedy_Going_Deeper_With_2015_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>Very deep convolutional networks for large-scale image
recognition</strong> (2014), K. Simonyan and A. Zisserman <a
href="http://arxiv.org/pdf/1409.1556">[pdf]</a></li>
<li><strong>Return of the devil in the details: delving deep into
convolutional nets</strong> (2014), K. Chatfield et al. <a
href="http://arxiv.org/pdf/1405.3531">[pdf]</a></li>
<li><strong>OverFeat: Integrated recognition, localization and detection
using convolutional networks</strong> (2013), P. Sermanet et al. <a
href="http://arxiv.org/pdf/1312.6229">[pdf]</a></li>
<li><strong>Maxout networks</strong> (2013), I. Goodfellow et al. <a
href="http://arxiv.org/pdf/1302.4389v4">[pdf]</a></li>
<li><strong>Network in network</strong> (2013), M. Lin et al. <a
href="http://arxiv.org/pdf/1312.4400">[pdf]</a></li>
<li><strong>ImageNet classification with deep convolutional neural
networks</strong> (2012), A. Krizhevsky et al. <a
href="http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf">[pdf]</a></li>
</ul>
<!---[Key researchers]  [Christian Szegedy](https://scholar.google.ca/citations?hl=en&user=3QeF7mAAAAAJ), [Kaming He](https://scholar.google.ca/citations?hl=en&user=DhtAFkwAAAAJ), [Shaoqing Ren](https://scholar.google.ca/citations?hl=en&user=AUhj438AAAAJ), [Jian Sun](https://scholar.google.ca/citations?hl=en&user=ALVSZAYAAAAJ), [Geoffrey Hinton](https://scholar.google.ca/citations?user=JicYPdAAAAAJ), [Yoshua Bengio](https://scholar.google.ca/citations?user=kukA0LcAAAAJ), [Yann LeCun](https://scholar.google.ca/citations?hl=en&user=WLN3QrAAAAAJ)-->
<h3 id="image-segmentation-object-detection">Image: Segmentation /
Object Detection</h3>
<ul>
<li><strong>You only look once: Unified, real-time object
detection</strong> (2016), J. Redmon et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Redmon_You_Only_Look_CVPR_2016_paper.pdf">[pdf]</a></li>
<li><strong>Fully convolutional networks for semantic
segmentation</strong> (2015), J. Long et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Long_Fully_Convolutional_Networks_2015_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>Faster R-CNN: Towards Real-Time Object Detection with Region
Proposal Networks</strong> (2015), S. Ren et al. <a
href="http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf">[pdf]</a></li>
<li><strong>Fast R-CNN</strong> (2015), R. Girshick <a
href="http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Girshick_Fast_R-CNN_ICCV_2015_paper.pdf">[pdf]</a></li>
<li><strong>Rich feature hierarchies for accurate object detection and
semantic segmentation</strong> (2014), R. Girshick et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Girshick_Rich_Feature_Hierarchies_2014_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>Spatial pyramid pooling in deep convolutional networks for
visual recognition</strong> (2014), K. He et al. <a
href="http://arxiv.org/pdf/1406.4729">[pdf]</a></li>
<li><strong>Semantic image segmentation with deep convolutional nets and
fully connected CRFs</strong>, L. Chen et al. <a
href="https://arxiv.org/pdf/1412.7062">[pdf]</a></li>
<li><strong>Learning hierarchical features for scene labeling</strong>
(2013), C. Farabet et al. <a
href="https://hal-enpc.archives-ouvertes.fr/docs/00/74/20/77/PDF/farabet-pami-13.pdf">[pdf]</a></li>
</ul>
<!---[Key researchers]  [Ross Girshick](https://scholar.google.ca/citations?hl=en&user=W8VIEZgAAAAJ), [Jeff Donahue](https://scholar.google.ca/citations?hl=en&user=UfbuDH8AAAAJ), [Trevor Darrell](https://scholar.google.ca/citations?hl=en&user=bh-uRFMAAAAJ)-->
<h3 id="image-video-etc">Image / Video / Etc</h3>
<ul>
<li><strong>Image Super-Resolution Using Deep Convolutional
Networks</strong> (2016), C. Dong et al. <a
href="https://arxiv.org/pdf/1501.00092v3.pdf">[pdf]</a></li>
<li><strong>A neural algorithm of artistic style</strong> (2015), L.
Gatys et al. <a href="https://arxiv.org/pdf/1508.06576">[pdf]</a></li>
<li><strong>Deep visual-semantic alignments for generating image
descriptions</strong> (2015), A. Karpathy and L. Fei-Fei <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Karpathy_Deep_Visual-Semantic_Alignments_2015_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>Show, attend and tell: Neural image caption generation with
visual attention</strong> (2015), K. Xu et al. <a
href="http://arxiv.org/pdf/1502.03044">[pdf]</a></li>
<li><strong>Show and tell: A neural image caption generator</strong>
(2015), O. Vinyals et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Vinyals_Show_and_Tell_2015_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>Long-term recurrent convolutional networks for visual
recognition and description</strong> (2015), J. Donahue et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Donahue_Long-Term_Recurrent_Convolutional_2015_CVPR_paper.pdf">[pdf]</a></li>
<li><strong>VQA: Visual question answering</strong> (2015), S. Antol et
al. <a
href="http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Antol_VQA_Visual_Question_ICCV_2015_paper.pdf">[pdf]</a></li>
<li><strong>DeepFace: Closing the gap to human-level performance in face
verification</strong> (2014), Y. Taigman et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Taigman_DeepFace_Closing_the_2014_CVPR_paper.pdf">[pdf]</a>:</li>
<li><strong>Large-scale video classification with convolutional neural
networks</strong> (2014), A. Karpathy et al. <a
href="http://vision.stanford.edu/pdf/karpathy14.pdf">[pdf]</a></li>
<li><strong>Two-stream convolutional networks for action recognition in
videos</strong> (2014), K. Simonyan et al. <a
href="http://papers.nips.cc/paper/5353-two-stream-convolutional-networks-for-action-recognition-in-videos.pdf">[pdf]</a></li>
<li><strong>3D convolutional neural networks for human action
recognition</strong> (2013), S. Ji et al. <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/icml2010_JiXYY10.pdf">[pdf]</a></li>
</ul>
<!---[Key researchers]  [Oriol Vinyals](https://scholar.google.ca/citations?user=NkzyCvUAAAAJ), [Andrej Karpathy](https://scholar.google.ca/citations?user=l8WuQJgAAAAJ)-->
<!---[Key researchers]  [Alex Graves](https://scholar.google.ca/citations?user=DaFHynwAAAAJ)-->
<h3 id="natural-language-processing-rnns">Natural Language Processing /
RNNs</h3>
<ul>
<li><strong>Neural Architectures for Named Entity Recognition</strong>
(2016), G. Lample et al. <a
href="http://aclweb.org/anthology/N/N16/N16-1030.pdf">[pdf]</a></li>
<li><strong>Exploring the limits of language modeling</strong> (2016),
R. Jozefowicz et al. <a
href="http://arxiv.org/pdf/1602.02410">[pdf]</a></li>
<li><strong>Teaching machines to read and comprehend</strong> (2015), K.
Hermann et al. <a
href="http://papers.nips.cc/paper/5945-teaching-machines-to-read-and-comprehend.pdf">[pdf]</a></li>
<li><strong>Effective approaches to attention-based neural machine
translation</strong> (2015), M. Luong et al. <a
href="https://arxiv.org/pdf/1508.04025">[pdf]</a></li>
<li><strong>Conditional random fields as recurrent neural
networks</strong> (2015), S. Zheng and S. Jayasumana. <a
href="http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Zheng_Conditional_Random_Fields_ICCV_2015_paper.pdf">[pdf]</a></li>
<li><strong>Memory networks</strong> (2014), J. Weston et al. <a
href="https://arxiv.org/pdf/1410.3916">[pdf]</a></li>
<li><strong>Neural turing machines</strong> (2014), A. Graves et al. <a
href="https://arxiv.org/pdf/1410.5401">[pdf]</a></li>
<li><strong>Neural machine translation by jointly learning to align and
translate</strong> (2014), D. Bahdanau et al. <a
href="http://arxiv.org/pdf/1409.0473">[pdf]</a></li>
<li><strong>Sequence to sequence learning with neural networks</strong>
(2014), I. Sutskever et al. <a
href="http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf">[pdf]</a></li>
<li><strong>Learning phrase representations using RNN encoder-decoder
for statistical machine translation</strong> (2014), K. Cho et al. <a
href="http://arxiv.org/pdf/1406.1078">[pdf]</a></li>
<li><strong>A convolutional neural network for modeling
sentences</strong> (2014), N. Kalchbrenner et al. <a
href="http://arxiv.org/pdf/1404.2188v1">[pdf]</a></li>
<li><strong>Convolutional neural networks for sentence
classification</strong> (2014), Y. Kim <a
href="http://arxiv.org/pdf/1408.5882">[pdf]</a></li>
<li><strong>Glove: Global vectors for word representation</strong>
(2014), J. Pennington et al. <a
href="http://anthology.aclweb.org/D/D14/D14-1162.pdf">[pdf]</a></li>
<li><strong>Distributed representations of sentences and
documents</strong> (2014), Q. Le and T. Mikolov <a
href="http://arxiv.org/pdf/1405.4053">[pdf]</a></li>
<li><strong>Distributed representations of words and phrases and their
compositionality</strong> (2013), T. Mikolov et al. <a
href="http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf">[pdf]</a></li>
<li><strong>Efficient estimation of word representations in vector
space</strong> (2013), T. Mikolov et al. <a
href="http://arxiv.org/pdf/1301.3781">[pdf]</a></li>
<li><strong>Recursive deep models for semantic compositionality over a
sentiment treebank</strong> (2013), R. Socher et al. <a
href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.383.1327&amp;rep=rep1&amp;type=pdf">[pdf]</a></li>
<li><strong>Generating sequences with recurrent neural networks</strong>
(2013), A. Graves. <a
href="https://arxiv.org/pdf/1308.0850">[pdf]</a></li>
</ul>
<!---[Key researchers]  [Kyunghyun Cho](https://scholar.google.ca/citations?user=0RAmmIAAAAAJ), [Oriol Vinyals](https://scholar.google.ca/citations?user=NkzyCvUAAAAJ), [Richard Socher](https://scholar.google.ca/citations?hl=en&user=FaOcyfMAAAAJ), [Tomas Mikolov](https://scholar.google.ca/citations?user=oBu8kMMAAAAJ), [Christopher D. Manning](https://scholar.google.ca/citations?user=1zmDOdwAAAAJ), [Yoshua Bengio](https://scholar.google.ca/citations?user=kukA0LcAAAAJ)-->
<h3 id="speech-other-domain">Speech / Other Domain</h3>
<ul>
<li><strong>End-to-end attention-based large vocabulary speech
recognition</strong> (2016), D. Bahdanau et al. <a
href="https://arxiv.org/pdf/1508.04395">[pdf]</a></li>
<li><strong>Deep speech 2: End-to-end speech recognition in English and
Mandarin</strong> (2015), D. Amodei et al. <a
href="https://arxiv.org/pdf/1512.02595">[pdf]</a></li>
<li><strong>Speech recognition with deep recurrent neural
networks</strong> (2013), A. Graves <a
href="http://arxiv.org/pdf/1303.5778.pdf">[pdf]</a></li>
<li><strong>Deep neural networks for acoustic modeling in speech
recognition: The shared views of four research groups</strong> (2012),
G. Hinton et al. <a
href="http://www.cs.toronto.edu/~asamir/papers/SPM_DNN_12.pdf">[pdf]</a></li>
<li><strong>Context-dependent pre-trained deep neural networks for
large-vocabulary speech recognition</strong> (2012) G. Dahl et al. <a
href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.337.7548&amp;rep=rep1&amp;type=pdf">[pdf]</a></li>
<li><strong>Acoustic modeling using deep belief networks</strong>
(2012), A. Mohamed et al. <a
href="http://www.cs.toronto.edu/~asamir/papers/speechDBN_jrnl.pdf">[pdf]</a></li>
</ul>
<!---[Key researchers]  [Alex Graves](https://scholar.google.ca/citations?user=DaFHynwAAAAJ), [Geoffrey Hinton](https://scholar.google.ca/citations?user=JicYPdAAAAAJ), [Dong Yu](https://scholar.google.ca/citations?hl=en&user=tMY31_gAAAAJ)-->
<h3 id="reinforcement-learning-robotics">Reinforcement Learning /
Robotics</h3>
<ul>
<li><strong>End-to-end training of deep visuomotor policies</strong>
(2016), S. Levine et al. <a
href="http://www.jmlr.org/papers/volume17/15-522/source/15-522.pdf">[pdf]</a></li>
<li><strong>Learning Hand-Eye Coordination for Robotic Grasping with
Deep Learning and Large-Scale Data Collection</strong> (2016), S. Levine
et al. <a href="https://arxiv.org/pdf/1603.02199">[pdf]</a></li>
<li><strong>Asynchronous methods for deep reinforcement
learning</strong> (2016), V. Mnih et al. <a
href="http://www.jmlr.org/proceedings/papers/v48/mniha16.pdf">[pdf]</a></li>
<li><strong>Deep Reinforcement Learning with Double Q-Learning</strong>
(2016), H. Hasselt et al. <a
href="https://arxiv.org/pdf/1509.06461.pdf">[pdf]</a></li>
<li><strong>Mastering the game of Go with deep neural networks and tree
search</strong> (2016), D. Silver et al. <a
href="http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html">[pdf]</a></li>
<li><strong>Continuous control with deep reinforcement learning</strong>
(2015), T. Lillicrap et al. <a
href="https://arxiv.org/pdf/1509.02971">[pdf]</a></li>
<li><strong>Human-level control through deep reinforcement
learning</strong> (2015), V. Mnih et al. <a
href="http://www.davidqiu.com:8888/research/nature14236.pdf">[pdf]</a></li>
<li><strong>Deep learning for detecting robotic grasps</strong> (2015),
I. Lenz et al. <a
href="http://www.cs.cornell.edu/~asaxena/papers/lenz_lee_saxena_deep_learning_grasping_ijrr2014.pdf">[pdf]</a></li>
<li><strong>Playing atari with deep reinforcement learning</strong>
(2013), V. Mnih et al. <a
href="http://arxiv.org/pdf/1312.5602.pdf">[pdf]</a>)</li>
</ul>
<!---[Key researchers]  [Sergey Levine](https://scholar.google.ca/citations?user=8R35rCwAAAAJ), [Volodymyr Mnih](https://scholar.google.ca/citations?hl=en&user=rLdfJ1gAAAAJ), [David Silver](https://scholar.google.ca/citations?user=-8DNE4UAAAAJ)-->
<h3 id="more-papers-from-2016">More Papers from 2016</h3>
<ul>
<li><strong>Layer Normalization</strong> (2016), J. Ba et al. <a
href="https://arxiv.org/pdf/1607.06450v1.pdf">[pdf]</a></li>
<li><strong>Learning to learn by gradient descent by gradient
descent</strong> (2016), M. Andrychowicz et al. <a
href="http://arxiv.org/pdf/1606.04474v1">[pdf]</a></li>
<li><strong>Domain-adversarial training of neural networks</strong>
(2016), Y. Ganin et al. <a
href="http://www.jmlr.org/papers/volume17/15-239/source/15-239.pdf">[pdf]</a></li>
<li><strong>WaveNet: A Generative Model for Raw Audio</strong> (2016),
A. Oord et al. <a href="https://arxiv.org/pdf/1609.03499v2">[pdf]</a> <a
href="https://deepmind.com/blog/wavenet-generative-model-raw-audio/">[web]</a></li>
<li><strong>Colorful image colorization</strong> (2016), R. Zhang et
al. <a href="https://arxiv.org/pdf/1603.08511">[pdf]</a></li>
<li><strong>Generative visual manipulation on the natural image
manifold</strong> (2016), J. Zhu et al. <a
href="https://arxiv.org/pdf/1609.03552">[pdf]</a></li>
<li><strong>Texture networks: Feed-forward synthesis of textures and
stylized images</strong> (2016), D Ulyanov et al. <a
href="http://www.jmlr.org/proceedings/papers/v48/ulyanov16.pdf">[pdf]</a></li>
<li><strong>SSD: Single shot multibox detector</strong> (2016), W. Liu
et al. <a href="https://arxiv.org/pdf/1512.02325">[pdf]</a></li>
<li><strong>SqueezeNet: AlexNet-level accuracy with 50x fewer parameters
and&lt; 1MB model size</strong> (2016), F. Iandola et al. <a
href="http://arxiv.org/pdf/1602.07360">[pdf]</a></li>
<li><strong>Eie: Efficient inference engine on compressed deep neural
network</strong> (2016), S. Han et al. <a
href="http://arxiv.org/pdf/1602.01528">[pdf]</a></li>
<li><strong>Binarized neural networks: Training deep neural networks
with weights and activations constrained to+ 1 or-1</strong> (2016), M.
Courbariaux et al. <a
href="https://arxiv.org/pdf/1602.02830">[pdf]</a></li>
<li><strong>Dynamic memory networks for visual and textual question
answering</strong> (2016), C. Xiong et al. <a
href="http://www.jmlr.org/proceedings/papers/v48/xiong16.pdf">[pdf]</a></li>
<li><strong>Stacked attention networks for image question
answering</strong> (2016), Z. Yang et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Yang_Stacked_Attention_Networks_CVPR_2016_paper.pdf">[pdf]</a></li>
<li><strong>Hybrid computing using a neural network with dynamic
external memory</strong> (2016), A. Graves et al. <a
href="https://www.gwern.net/docs/2016-graves.pdf">[pdf]</a></li>
<li><strong>Google’s neural machine translation system: Bridging the gap
between human and machine translation</strong> (2016), Y. Wu et al. <a
href="https://arxiv.org/pdf/1609.08144">[pdf]</a></li>
</ul>
<hr />
<h3 id="new-papers">New papers</h3>
<p><em>Newly published papers (&lt; 6 months) which are worth
reading</em> - MobileNets: Efficient Convolutional Neural Networks for
Mobile Vision Applications (2017), Andrew G. Howard et al. <a
href="https://arxiv.org/pdf/1704.04861.pdf">[pdf]</a> - Convolutional
Sequence to Sequence Learning (2017), Jonas Gehring et al. <a
href="https://arxiv.org/pdf/1705.03122">[pdf]</a> - A Knowledge-Grounded
Neural Conversation Model (2017), Marjan Ghazvininejad et al. <a
href="https://arxiv.org/pdf/1702.01932">[pdf]</a> - Accurate, Large
Minibatch SGD:Training ImageNet in 1 Hour (2017), Priya Goyal et al. <a
href="https://research.fb.com/wp-content/uploads/2017/06/imagenet1kin1h3.pdf">[pdf]</a>
- TACOTRON: Towards end-to-end speech synthesis (2017), Y. Wang et
al. <a href="https://arxiv.org/pdf/1703.10135.pdf">[pdf]</a> - Deep
Photo Style Transfer (2017), F. Luan et al. <a
href="http://arxiv.org/pdf/1703.07511v1.pdf">[pdf]</a> - Evolution
Strategies as a Scalable Alternative to Reinforcement Learning (2017),
T. Salimans et al. <a
href="http://arxiv.org/pdf/1703.03864v1.pdf">[pdf]</a> - Deformable
Convolutional Networks (2017), J. Dai et al. <a
href="http://arxiv.org/pdf/1703.06211v2.pdf">[pdf]</a> - Mask R-CNN
(2017), K. He et al. <a
href="https://128.84.21.199/pdf/1703.06870">[pdf]</a> - Learning to
discover cross-domain relations with generative adversarial networks
(2017), T. Kim et al. <a
href="http://arxiv.org/pdf/1703.05192v1.pdf">[pdf]</a> - Deep voice:
Real-time neural text-to-speech (2017), S. Arik et al., <a
href="http://arxiv.org/pdf/1702.07825v2.pdf">[pdf]</a> - PixelNet:
Representation of the pixels, by the pixels, and for the pixels (2017),
A. Bansal et al. <a
href="http://arxiv.org/pdf/1702.06506v1.pdf">[pdf]</a> - Batch
renormalization: Towards reducing minibatch dependence in
batch-normalized models (2017), S. Ioffe. <a
href="https://arxiv.org/abs/1702.03275">[pdf]</a> - Wasserstein GAN
(2017), M. Arjovsky et al. <a
href="https://arxiv.org/pdf/1701.07875v1">[pdf]</a> - Understanding deep
learning requires rethinking generalization (2017), C. Zhang et al. <a
href="https://arxiv.org/pdf/1611.03530">[pdf]</a> - Least squares
generative adversarial networks (2016), X. Mao et al. <a
href="https://arxiv.org/abs/1611.04076v2">[pdf]</a></p>
<h3 id="old-papers">Old Papers</h3>
<p><em>Classic papers published before 2012</em> - An analysis of
single-layer networks in unsupervised feature learning (2011), A. Coates
et al. <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2011_CoatesNL11.pdf">[pdf]</a>
- Deep sparse rectifier neural networks (2011), X. Glorot et al. <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2011_GlorotBB11.pdf">[pdf]</a>
- Natural language processing (almost) from scratch (2011), R. Collobert
et al. <a href="http://arxiv.org/pdf/1103.0398">[pdf]</a> - Recurrent
neural network based language model (2010), T. Mikolov et al. <a
href="http://www.fit.vutbr.cz/research/groups/speech/servite/2010/rnnlm_mikolov.pdf">[pdf]</a>
- Stacked denoising autoencoders: Learning useful representations in a
deep network with a local denoising criterion (2010), P. Vincent et
al. <a
href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.297.3484&amp;rep=rep1&amp;type=pdf">[pdf]</a>
- Learning mid-level features for recognition (2010), Y. Boureau <a
href="http://ece.duke.edu/~lcarin/boureau-cvpr-10.pdf">[pdf]</a> - A
practical guide to training restricted boltzmann machines (2010), G.
Hinton <a
href="http://www.csri.utoronto.ca/~hinton/absps/guideTR.pdf">[pdf]</a> -
Understanding the difficulty of training deep feedforward neural
networks (2010), X. Glorot and Y. Bengio <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_GlorotB10.pdf">[pdf]</a>
- Why does unsupervised pre-training help deep learning (2010), D. Erhan
et al. <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_ErhanCBV10.pdf">[pdf]</a>
- Learning deep architectures for AI (2009), Y. Bengio. <a
href="http://sanghv.com/download/soft/machine%20learning,%20artificial%20intelligence,%20mathematics%20ebooks/ML/learning%20deep%20architectures%20for%20AI%20(2009).pdf">[pdf]</a>
- Convolutional deep belief networks for scalable unsupervised learning
of hierarchical representations (2009), H. Lee et al. <a
href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.149.802&amp;rep=rep1&amp;type=pdf">[pdf]</a>
- Greedy layer-wise training of deep networks (2007), Y. Bengio et
al. <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_739.pdf">[pdf]</a>
- Reducing the dimensionality of data with neural networks, G. Hinton
and R. Salakhutdinov. <a
href="http://homes.mpimf-heidelberg.mpg.de/~mhelmsta/pdf/2006%20Hinton%20Salakhudtkinov%20Science.pdf">[pdf]</a>
- A fast learning algorithm for deep belief nets (2006), G. Hinton et
al. <a
href="http://nuyoo.utm.mx/~jjf/rna/A8%20A%20fast%20learning%20algorithm%20for%20deep%20belief%20nets.pdf">[pdf]</a>
- Gradient-based learning applied to document recognition (1998), Y.
LeCun et al. <a
href="http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf">[pdf]</a> -
Long short-term memory (1997), S. Hochreiter and J. Schmidhuber. <a
href="http://www.mitpressjournals.org/doi/pdfplus/10.1162/neco.1997.9.8.1735">[pdf]</a></p>
<h3 id="hw-sw-dataset">HW / SW / Dataset</h3>
<ul>
<li>SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016),
Rajpurkar et al. <a
href="https://arxiv.org/pdf/1606.05250.pdf">[pdf]</a></li>
<li>OpenAI gym (2016), G. Brockman et al. <a
href="https://arxiv.org/pdf/1606.01540">[pdf]</a></li>
<li>TensorFlow: Large-scale machine learning on heterogeneous
distributed systems (2016), M. Abadi et al. <a
href="http://arxiv.org/pdf/1603.04467">[pdf]</a></li>
<li>Theano: A Python framework for fast computation of mathematical
expressions, R. Al-Rfou et al.</li>
<li>Torch7: A matlab-like environment for machine learning, R. Collobert
et al. <a
href="https://ronan.collobert.com/pub/matos/2011_torch7_nipsw.pdf">[pdf]</a></li>
<li>MatConvNet: Convolutional neural networks for matlab (2015), A.
Vedaldi and K. Lenc <a
href="http://arxiv.org/pdf/1412.4564">[pdf]</a></li>
<li>Imagenet large scale visual recognition challenge (2015), O.
Russakovsky et al. <a
href="http://arxiv.org/pdf/1409.0575">[pdf]</a></li>
<li>Caffe: Convolutional architecture for fast feature embedding (2014),
Y. Jia et al. <a href="http://arxiv.org/pdf/1408.5093">[pdf]</a></li>
</ul>
<h3 id="book-survey-review">Book / Survey / Review</h3>
<ul>
<li>On the Origin of Deep Learning (2017), H. Wang and Bhiksha Raj. <a
href="https://arxiv.org/pdf/1702.07800">[pdf]</a></li>
<li>Deep Reinforcement Learning: An Overview (2017), Y. Li, <a
href="http://arxiv.org/pdf/1701.07274v2.pdf">[pdf]</a></li>
<li>Neural Machine Translation and Sequence-to-sequence Models(2017): A
Tutorial, G. Neubig. <a
href="http://arxiv.org/pdf/1703.01619v1.pdf">[pdf]</a></li>
<li>Neural Network and Deep Learning (Book, Jan 2017), Michael Nielsen.
<a
href="http://neuralnetworksanddeeplearning.com/index.html">[html]</a></li>
<li>Deep learning (Book, 2016), Goodfellow et al. <a
href="http://www.deeplearningbook.org/">[html]</a></li>
<li>LSTM: A search space odyssey (2016), K. Greff et al. <a
href="https://arxiv.org/pdf/1503.04069.pdf?utm_content=buffereddc5&amp;utm_medium=social&amp;utm_source=plus.google.com&amp;utm_campaign=buffer">[pdf]</a></li>
<li>Tutorial on Variational Autoencoders (2016), C. Doersch. <a
href="https://arxiv.org/pdf/1606.05908">[pdf]</a></li>
<li>Deep learning (2015), Y. LeCun, Y. Bengio and G. Hinton <a
href="https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf">[pdf]</a></li>
<li>Deep learning in neural networks: An overview (2015), J. Schmidhuber
<a href="http://arxiv.org/pdf/1404.7828">[pdf]</a></li>
<li>Representation learning: A review and new perspectives (2013), Y.
Bengio et al. <a href="http://arxiv.org/pdf/1206.5538">[pdf]</a></li>
</ul>
<h3 id="video-lectures-tutorials-blogs">Video Lectures / Tutorials /
Blogs</h3>
<p><em>(Lectures)</em> - CS231n, Convolutional Neural Networks for
Visual Recognition, Stanford University <a
href="http://cs231n.stanford.edu/">[web]</a> - CS224d, Deep Learning for
Natural Language Processing, Stanford University <a
href="http://cs224d.stanford.edu/">[web]</a> - Oxford Deep NLP 2017,
Deep Learning for Natural Language Processing, University of Oxford <a
href="https://github.com/oxford-cs-deepnlp-2017/lectures">[web]</a></p>
<p><em>(Tutorials)</em> - NIPS 2016 Tutorials, Long Beach <a
href="https://nips.cc/Conferences/2016/Schedule?type=Tutorial">[web]</a>
- ICML 2016 Tutorials, New York City <a
href="http://techtalks.tv/icml/2016/tutorials/">[web]</a> - ICLR 2016
Videos, San Juan <a
href="http://videolectures.net/iclr2016_san_juan/">[web]</a> - Deep
Learning Summer School 2016, Montreal <a
href="http://videolectures.net/deeplearning2016_montreal/">[web]</a> -
Bay Area Deep Learning School 2016, Stanford <a
href="https://www.bayareadlschool.org/">[web]</a></p>
<p><em>(Blogs)</em> - OpenAI <a href="https://www.openai.com/">[web]</a>
- Distill <a href="http://distill.pub/">[web]</a> - Andrej Karpathy Blog
<a href="http://karpathy.github.io/">[web]</a> - Colah’s Blog <a
href="http://colah.github.io/">[Web]</a> - WildML <a
href="http://www.wildml.com/">[Web]</a> - FastML <a
href="http://www.fastml.com/">[web]</a> - TheMorningPaper <a
href="https://blog.acolyer.org">[web]</a></p>
<h3 id="appendix-more-than-top-100">Appendix: More than Top 100</h3>
<p><em>(2016)</em> - A character-level decoder without explicit
segmentation for neural machine translation (2016), J. Chung et al. <a
href="https://arxiv.org/pdf/1603.06147">[pdf]</a> - Dermatologist-level
classification of skin cancer with deep neural networks (2017), A.
Esteva et al. <a
href="http://www.nature.com/nature/journal/v542/n7639/full/nature21056.html">[html]</a>
- Weakly supervised object localization with multi-fold multiple
instance learning (2017), R. Gokberk et al. <a
href="https://arxiv.org/pdf/1503.00949">[pdf]</a> - Brain tumor
segmentation with deep neural networks (2017), M. Havaei et al. <a
href="https://arxiv.org/pdf/1505.03540">[pdf]</a> - Professor Forcing: A
New Algorithm for Training Recurrent Networks (2016), A. Lamb et al. <a
href="https://arxiv.org/pdf/1610.09038">[pdf]</a> - Adversarially
learned inference (2016), V. Dumoulin et al. <a
href="https://ishmaelbelghazi.github.io/ALI/">[web]</a><a
href="https://arxiv.org/pdf/1606.00704v1">[pdf]</a> - Understanding
convolutional neural networks (2016), J. Koushik <a
href="https://arxiv.org/pdf/1605.09081v1">[pdf]</a> - Taking the human
out of the loop: A review of bayesian optimization (2016), B. Shahriari
et al. <a
href="https://www.cs.ox.ac.uk/people/nando.defreitas/publications/BayesOptLoop.pdf">[pdf]</a>
- Adaptive computation time for recurrent neural networks (2016), A.
Graves <a href="http://arxiv.org/pdf/1603.08983">[pdf]</a> - Densely
connected convolutional networks (2016), G. Huang et al. <a
href="https://arxiv.org/pdf/1608.06993v1">[pdf]</a> - Region-based
convolutional networks for accurate object detection and segmentation
(2016), R. Girshick et al.  - Continuous deep q-learning with
model-based acceleration (2016), S. Gu et al. <a
href="http://www.jmlr.org/proceedings/papers/v48/gu16.pdf">[pdf]</a> - A
thorough examination of the cnn/daily mail reading comprehension task
(2016), D. Chen et al. <a
href="https://arxiv.org/pdf/1606.02858">[pdf]</a> - Achieving open
vocabulary neural machine translation with hybrid word-character models,
M. Luong and C. Manning. <a
href="https://arxiv.org/pdf/1604.00788">[pdf]</a> - Very Deep
Convolutional Networks for Natural Language Processing (2016), A.
Conneau et al. <a href="https://arxiv.org/pdf/1606.01781">[pdf]</a> -
Bag of tricks for efficient text classification (2016), A. Joulin et
al. <a href="https://arxiv.org/pdf/1607.01759">[pdf]</a> - Efficient
piecewise training of deep structured models for semantic segmentation
(2016), G. Lin et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Lin_Efficient_Piecewise_Training_CVPR_2016_paper.pdf">[pdf]</a>
- Learning to compose neural networks for question answering (2016), J.
Andreas et al. <a href="https://arxiv.org/pdf/1601.01705">[pdf]</a> -
Perceptual losses for real-time style transfer and super-resolution
(2016), J. Johnson et al. <a
href="https://arxiv.org/pdf/1603.08155">[pdf]</a> - Reading text in the
wild with convolutional neural networks (2016), M. Jaderberg et al. <a
href="http://arxiv.org/pdf/1412.1842">[pdf]</a> - What makes for
effective detection proposals? (2016), J. Hosang et al. <a
href="https://arxiv.org/pdf/1502.05082">[pdf]</a> - Inside-outside net:
Detecting objects in context with skip pooling and recurrent neural
networks (2016), S. Bell et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Bell_Inside-Outside_Net_Detecting_CVPR_2016_paper.pdf">[pdf]</a>.
- Instance-aware semantic segmentation via multi-task network cascades
(2016), J. Dai et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Dai_Instance-Aware_Semantic_Segmentation_CVPR_2016_paper.pdf">[pdf]</a>
- Conditional image generation with pixelcnn decoders (2016), A. van den
Oord et al. <a
href="http://papers.nips.cc/paper/6527-tree-structured-reinforcement-learning-for-sequential-object-localization.pdf">[pdf]</a>
- Deep networks with stochastic depth (2016), G. Huang et al., <a
href="https://arxiv.org/pdf/1603.09382">[pdf]</a> - Consistency and
Fluctuations For Stochastic Gradient Langevin Dynamics (2016), Yee Whye
Teh et al. <a
href="http://www.jmlr.org/papers/volume17/teh16a/teh16a.pdf">[pdf]</a></p>
<p><em>(2015)</em> - Ask your neurons: A neural-based approach to
answering questions about images (2015), M. Malinowski et al. <a
href="http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Malinowski_Ask_Your_Neurons_ICCV_2015_paper.pdf">[pdf]</a>
- Exploring models and data for image question answering (2015), M. Ren
et al. <a
href="http://papers.nips.cc/paper/5640-stochastic-variational-inference-for-hidden-markov-models.pdf">[pdf]</a>
- Are you talking to a machine? dataset and methods for multilingual
image question (2015), H. Gao et al. <a
href="http://papers.nips.cc/paper/5641-are-you-talking-to-a-machine-dataset-and-methods-for-multilingual-image-question.pdf">[pdf]</a>
- Mind’s eye: A recurrent visual representation for image caption
generation (2015), X. Chen and C. Zitnick. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Chen_Minds_Eye_A_2015_CVPR_paper.pdf">[pdf]</a>
- From captions to visual concepts and back (2015), H. Fang et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Fang_From_Captions_to_2015_CVPR_paper.pdf">[pdf]</a>.
- Towards AI-complete question answering: A set of prerequisite toy
tasks (2015), J. Weston et al. <a
href="http://arxiv.org/pdf/1502.05698">[pdf]</a> - Ask me anything:
Dynamic memory networks for natural language processing (2015), A. Kumar
et al. <a href="http://arxiv.org/pdf/1506.07285">[pdf]</a> -
Unsupervised learning of video representations using LSTMs (2015), N.
Srivastava et al. <a
href="http://www.jmlr.org/proceedings/papers/v37/srivastava15.pdf">[pdf]</a>
- Deep compression: Compressing deep neural networks with pruning,
trained quantization and huffman coding (2015), S. Han et al. <a
href="https://arxiv.org/pdf/1510.00149">[pdf]</a> - Improved semantic
representations from tree-structured long short-term memory networks
(2015), K. Tai et al. <a
href="https://arxiv.org/pdf/1503.00075">[pdf]</a> - Character-aware
neural language models (2015), Y. Kim et al. <a
href="https://arxiv.org/pdf/1508.06615">[pdf]</a> - Grammar as a foreign
language (2015), O. Vinyals et al. <a
href="http://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf">[pdf]</a>
- Trust Region Policy Optimization (2015), J. Schulman et al. <a
href="http://www.jmlr.org/proceedings/papers/v37/schulman15.pdf">[pdf]</a>
- Beyond short snippents: Deep networks for video classification (2015)
<a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Ng_Beyond_Short_Snippets_2015_CVPR_paper.pdf">[pdf]</a>
- Learning Deconvolution Network for Semantic Segmentation (2015), H.
Noh et al. <a href="https://arxiv.org/pdf/1505.04366v1">[pdf]</a> -
Learning spatiotemporal features with 3d convolutional networks (2015),
D. Tran et al. <a
href="http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Tran_Learning_Spatiotemporal_Features_ICCV_2015_paper.pdf">[pdf]</a>
- Understanding neural networks through deep visualization (2015), J.
Yosinski et al. <a href="https://arxiv.org/pdf/1506.06579">[pdf]</a> -
An Empirical Exploration of Recurrent Network Architectures (2015), R.
Jozefowicz et al. <a
href="http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf">[pdf]</a>
- Deep generative image models using a laplacian pyramid of adversarial
networks (2015), E.Denton et al. <a
href="http://papers.nips.cc/paper/5773-deep-generative-image-models-using-a-laplacian-pyramid-of-adversarial-networks.pdf">[pdf]</a>
- Gated Feedback Recurrent Neural Networks (2015), J. Chung et al. <a
href="http://www.jmlr.org/proceedings/papers/v37/chung15.pdf">[pdf]</a>
- Fast and accurate deep network learning by exponential linear units
(ELUS) (2015), D. Clevert et al. <a
href="https://arxiv.org/pdf/1511.07289.pdf%5Cnhttp://arxiv.org/abs/1511.07289%5Cnhttp://arxiv.org/abs/1511.07289">[pdf]</a>
- Pointer networks (2015), O. Vinyals et al. <a
href="http://papers.nips.cc/paper/5866-pointer-networks.pdf">[pdf]</a> -
Visualizing and Understanding Recurrent Networks (2015), A. Karpathy et
al. <a href="https://arxiv.org/pdf/1506.02078">[pdf]</a> -
Attention-based models for speech recognition (2015), J. Chorowski et
al. <a
href="http://papers.nips.cc/paper/5847-attention-based-models-for-speech-recognition.pdf">[pdf]</a>
- End-to-end memory networks (2015), S. Sukbaatar et al. <a
href="http://papers.nips.cc/paper/5846-end-to-end-memory-networks.pdf">[pdf]</a>
- Describing videos by exploiting temporal structure (2015), L. Yao et
al. <a
href="http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Yao_Describing_Videos_by_ICCV_2015_paper.pdf">[pdf]</a>
- A neural conversational model (2015), O. Vinyals and Q. Le. <a
href="https://arxiv.org/pdf/1506.05869.pdf">[pdf]</a> - Improving
distributional similarity with lessons learned from word embeddings, O.
Levy et al. [[pdf]]
(https://www.transacl.org/ojs/index.php/tacl/article/download/570/124) -
Transition-Based Dependency Parsing with Stack Long Short-Term Memory
(2015), C. Dyer et al. <a
href="http://aclweb.org/anthology/P/P15/P15-1033.pdf">[pdf]</a> -
Improved Transition-Based Parsing by Modeling Characters instead of
Words with LSTMs (2015), M. Ballesteros et al. <a
href="http://aclweb.org/anthology/D/D15/D15-1041.pdf">[pdf]</a> -
Finding function in form: Compositional character models for open
vocabulary word representation (2015), W. Ling et al. <a
href="http://aclweb.org/anthology/D/D15/D15-1176.pdf">[pdf]</a></p>
<p><em>(~2014)</em> - DeepPose: Human pose estimation via deep neural
networks (2014), A. Toshev and C. Szegedy <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Toshev_DeepPose_Human_Pose_2014_CVPR_paper.pdf">[pdf]</a>
- Learning a Deep Convolutional Network for Image Super-Resolution
(2014, C. Dong et al. <a
href="https://www.researchgate.net/profile/Chen_Change_Loy/publication/264552416_Lecture_Notes_in_Computer_Science/links/53e583e50cf25d674e9c280e.pdf">[pdf]</a>
- Recurrent models of visual attention (2014), V. Mnih et al. <a
href="http://arxiv.org/pdf/1406.6247.pdf">[pdf]</a> - Empirical
evaluation of gated recurrent neural networks on sequence modeling
(2014), J. Chung et al. <a
href="https://arxiv.org/pdf/1412.3555">[pdf]</a> - Addressing the rare
word problem in neural machine translation (2014), M. Luong et al. <a
href="https://arxiv.org/pdf/1410.8206">[pdf]</a> - On the properties of
neural machine translation: Encoder-decoder approaches (2014), K. Cho
et. al. - Recurrent neural network regularization (2014), W. Zaremba et
al. <a href="http://arxiv.org/pdf/1409.2329">[pdf]</a> - Intriguing
properties of neural networks (2014), C. Szegedy et al. <a
href="https://arxiv.org/pdf/1312.6199.pdf">[pdf]</a> - Towards
end-to-end speech recognition with recurrent neural networks (2014), A.
Graves and N. Jaitly. <a
href="http://www.jmlr.org/proceedings/papers/v32/graves14.pdf">[pdf]</a>
- Scalable object detection using deep neural networks (2014), D. Erhan
et al. <a
href="http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Erhan_Scalable_Object_Detection_2014_CVPR_paper.pdf">[pdf]</a>
- On the importance of initialization and momentum in deep learning
(2013), I. Sutskever et al. <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/icml2013_sutskever13.pdf">[pdf]</a>
- Regularization of neural networks using dropconnect (2013), L. Wan et
al. <a
href="http://machinelearning.wustl.edu/mlpapers/paper_files/icml2013_wan13.pdf">[pdf]</a>
- Learning Hierarchical Features for Scene Labeling (2013), C. Farabet
et al. <a
href="https://hal-enpc.archives-ouvertes.fr/docs/00/74/20/77/PDF/farabet-pami-13.pdf">[pdf]</a>
- Linguistic Regularities in Continuous Space Word Representations
(2013), T. Mikolov et al. <a
href="http://www.aclweb.org/anthology/N13-1#page=784">[pdf]</a> - Large
scale distributed deep networks (2012), J. Dean et al. <a
href="http://papers.nips.cc/paper/4687-large-scale-distributed-deep-networks.pdf">[pdf]</a>
- A Fast and Accurate Dependency Parser using Neural Networks. Chen and
Manning. <a
href="http://cs.stanford.edu/people/danqi/papers/emnlp2014.pdf">[pdf]</a></p>
<h2 id="acknowledgement">Acknowledgement</h2>
<p>Thank you for all your contributions. Please make sure to read the <a
href="https://github.com/terryum/awesome-deep-learning-papers/blob/master/Contributing.md">contributing
guide</a> before you make a pull request.</p>
<h2 id="license">License</h2>
<p><a href="https://creativecommons.org/publicdomain/zero/1.0/"><img
src="http://mirrors.creativecommons.org/presskit/buttons/88x31/svg/cc-zero.svg"
alt="CC0" /></a></p>
<p>To the extent possible under law, <a
href="https://www.facebook.com/terryum.io/">Terry T. Um</a> has waived
all copyright and related or neighboring rights to this work.</p>
<p><a
href="https://github.com/terryum/awesome-deep-learning-papers">deeplearningpapers.md
Github</a></p>