Research in General

• How to write a great research paper

Basics of machine learning

• http://www.iro.umontreal.ca/~bengioy/DLbook/math.html

• http://www.iro.umontreal.ca/~bengioy/DLbook/ml.html

Basics of deep learning

• http://www.iro.umontreal.ca/~bengioy/DLbook/intro.html

• http://www.iro.umontreal.ca/~bengioy/DLbook/mlp.html

• Learning deep architectures for AI

• Practical recommendations for gradient-based training of deep architectures

• Quick’n’dirty introduction to deep learning: Advances in Deep Learning

• A fast learning algorithm for deep belief nets

• Greedy Layer-Wise Training of Deep Networks

• Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion

• Contractive auto-encoders: Explicit invariance during feature extraction

• Why does unsupervised pre-training help deep learning?

• An Analysis of Single Layer Networks in Unsupervised Feature Learning

• The importance of Encoding Versus Training With Sparse Coding and Vector Quantization

• Representation Learning: A Review and New Perspectives

• Deep Learning of Representations: Looking Forward

• Measuring Invariances in Deep Networks

• Neural networks course at USherbrooke [youtube]

Feedforward nets

• http://www.iro.umontreal.ca/~bengioy/DLbook/mlp.html

• “Improving Neural Nets with Dropout” by Nitish Srivastava

• “Deep Sparse Rectifier Neural Networks”

• “What is the best multi-stage architecture for object recognition?”

• “Maxout Networks”

MCMC

• Iain Murray’s MLSS slides

• Radford Neal’s Review Paper (old but still very comprehensive)

• Better Mixing via Deep Representations

Restricted Boltzmann Machines

• Unsupervised learning of distributions of binary vectors using 2-layer networks

• A practical guide to training restricted Boltzmann machines

• Training restricted Boltzmann machines using approximations to the likelihood gradient

• Tempered Markov Chain Monte Carlo for training of Restricted Boltzmann Machine

• How to Center Binary Restricted Boltzmann Machines

• Enhanced Gradient for Training Restricted Boltzmann Machines

• Using fast weights to improve persistent contrastive divergence

• Training Products of Experts by Minimizing Contrastive Divergence

Boltzmann Machines

• Deep Boltzmann Machines (Salakhutdinov & Hinton)

• Multimodal Learning with Deep Boltzmann Machines

• Multi-Prediction Deep Boltzmann Machines

• A Two-stage Pretraining Algorithm for Deep Boltzmann Machines

Regularized Auto-Encoders

• The Manifold Tangent Classifier

Regularization

Stochastic Nets & GSNs

• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

• Learning Stochastic Feedforward Neural Networks

• Generalized Denoising Auto-Encoders as Generative Models

• Deep Generative Stochastic Networks Trainable by Backprop

Others

• Slow, Decorrelated Features for Pretraining Complex Cell-like Networks

• What Regularized Auto-Encoders Learn from the Data Generating Distribution

• Generalized Denoising Auto-Encoders as Generative Models

• Why the logistic function?

Recurrent Nets

• Learning long-term dependencies with gradient descent is difficult

• Advances in Optimizing Recurrent Networks

• Learning recurrent neural networks with Hessian-free optimization

• On the importance of momentum and initialization in deep learning,

• Long short-term memory (Hochreiter & Schmidhuber)

• Generating Sequences With Recurrent Neural Networks

• Long Short-Term Memory in Echo State Networks: Details of a Simulation Study

• The "echo state" approach to analysing and training recurrent neural networks

• Backpropagation-Decorrelation: online recurrent learning with O(N) complexity

• New results on recurrent network training:Unifying the algorithms and accelerating convergence

• Audio Chord Recognition with Recurrent Neural Networks

• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

Convolutional Nets

• http://www.iro.umontreal.ca/~bengioy/DLbook/convnets.html

• Generalization and Network Design Strategies (LeCun)

• ImageNet Classification with Deep Convolutional Neural Networks, Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, NIPS 2012.

• On Random Weights and Unsupervised Feature Learning

Optimization issues with DL

• Curriculum Learning

• Evolving Culture vs Local Minima

• Knowledge Matters: Importance of Prior Information for Optimization

• Efficient Backprop

• Practical recommendations for gradient-based training of deep architectures

• Natural Gradient Works Efficiently (Amari 1998)

• Hessian Free

• Natural Gradient (TONGA)

• Revisiting Natural Gradient

NLP + DL

• Natural Language Processing (Almost) from Scratch

• DeViSE: A Deep Visual-Semantic Embedding Model

• Distributed Representations of Words and Phrases and their Compositionality

• Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection

CV+RBM

• Fields of Experts

• What makes a good model of natural images?

• Phone Recognition with the mean-covariance restricted Boltzmann machine

• Unsupervised Models of Images by Spike-and-Slab RBMs

CV + DL

• Imagenet classification with deep convolutional neural networks

• Learning to relate images

Scaling Up

• Large Scale Distributed Deep Networks

• Random search for hyper-parameter optimization

• Practical Bayesian Optimization of Machine Learning Algorithms

DL + Reinforcement learning

• Playing Atari with Deep Reinforcement Learning (paper not officially released yet!)

Graphical Models Background

• An Introduction to Graphical Models (Mike Jordan, brief course notes)

• A View of the EM Algorithm that Justifies Incremental, Sparse and Other Variants (Neal & Hinton, important paper to the modern understanding of Expectation-Maximization)

• A Unifying Review of Linear Gaussian Models (Roweis & Ghahramani, ties together PCA, factor analysis, hidden Markov models, Gaussian mixtures, k-means, linear dynamical systems)

• An Introduction to Variational Methods for Graphical Models (Jordan et al, mean-field, etc.)

Writing

• Writing a great research paper (video of the presentation)

Software documentation

• Python, Theano, Pylearn2, Linux (bash) (at least the 5 first sections), git (5 first sections), github/contributing to it (Theano doc), vim tutorial or emacs tutorial

Software lists of built-in commands/functions

• Bash commands

• List of Built-in Python Functions

• vim commands

Other Software stuff to know about:

• screen

• ssh

• ipython

• matplotlib

最新文章

1. Maven修改本地仓库路径
2. Apache Commons CLI命令行启动
3. Java Decompiler 反编译工具下载地址及JD-Eclipse设置菜单翻译
4. 大数据导致DataReader.Close超时的异常
5. 分享我收集的引擎、图形学、WebGL方面的电子资料
6. SharePoint 2013 开发——开发并部署webpart
7. fdquery update
8. 152. Maximum Product Subarray
9. ES6笔记-正则表达式和字符串正则方法
10. WIA Property Constant Definitions
11. 在一个apk中调用另外一个apk中的activity
12. c#操作sqlite
13. Php导出百万数据的优化
14. SEED实验——Environment Variable and Set-UID Program实验描述与实验任务
15. aspectj eclipse4.6下载地址
16. [CF52C]Circular RMQ【线段树】
17. net core体系-web应用程序-4net core2.0大白话带你入门-8asp.net core 内置DI容器（DependencyInjection，控制翻转）的一点小理解
18. Python内置类型(5)--迭代器类型
19. rsync实现数据同步
20. A2W，W2A等的使用

热门文章

1. 58、synchronized同步方法
2. Dream------scala--开发环境搭建
3. 【codeforces】【比赛题解】#920 Educational CF Round 37
4. 使用Python扫描网络MAC地址对应的IP地址
5. 007_苹果Mac系统锁屏不待机效果设置方法介绍
6. 五、springcloud之客户端负载均衡Ribbon
7. 如何从TFS（Visual Studio Team Foundation Server）映射下载本地文件夹
8. IOS使用xcode编译代码
9. java基础74 XML解析中的SAX解析相关知识点（网页知识）
10. JS中精选this关键字的指向规律你记住了吗