个人技术栈构建

Resources

Deep Learning

• Ian Goodfellow. Deep Learning Book
• CS230 Deep Learning

Natural Language Processing

• Dan Jurafsky. Speech and Language Processing (3rd)
• Yoav Goldberg. Neural Network Methods for Natural Language Processing
• Jacob Eisenstein. Natural Language Processing
• CS224n: Natural Language Processing with Deep Learning
• CS11-747, Neural Networks for NLP

Reinforcement Learning

• David Silver. Course on Reiforcement Learning
• CS234: Reinforcement Learning Winter 2019
• Deep Reinforcement Learning: An Overview
• Spinning Up in Deep RL produced by OpenAI

Machine Learning

• 李航. 《统计学习方法》
• Christopher Bishop. Pattern Recognition and Machine Learning

---

Natural Language Processing 学习

接下来成为专家的领域：Transformer + TL -> 知识图谱 -> DRL

Topic	cs224n	slp3	CS11-747	others
nlp basics: math and optimizers	lecture 0
Word Vectors	lecture 1: Introduction and Word Vectors lecture 2: Word Vectors 2 and Word Senses lecture 12: Information from parts of words: Subword Models	chapter 6: Vector Semantics	Distributional Semantics and Word Vectors (1/22/2019)	ruder.io/word-embeddings
Neural Networks	lecture 3: Word Window Classification, Neural Networks, and Matrix Calculus lecture 4: Backpropagation and Computation Graphs	chapter 7: Neural Networks and Neural LM		Neural Networks and Deep Learning
RNN and Language Models	lecture 6: Recurrent Neural Networks and Language Models lecture 7: Vanishing Gradients, Fancy RNNs	chapter 9: Sequence Processing with Recurrent Networks	A Simple (?) Exercise: Predicting the Next Word in a Sentence (1/17/2019) Recurrent Networks for Sentence or Language Modeling (1/29/2019)	Recurrent Neural Networks Tutorial, Part 1–Introduction to RNNs Understanding LSTM Networks The Unreasonable Effectiveness of Recurrent Neural Networks
seq2seq+Attention	lecture 8: Machine Translation, Seq2Seq and Attention	chapter 22: Machine Translation	Conditioned Generation (2/5/2019) Attention (2/7/2019)	https://github.com/tensorflow/nmt https://arxiv.org/abs/1703.01619
CNN for Text	lecture 11: ConvNets for NLP		Convolutional Neural Nets for Text (1/24/2019)
Contextual	lecture 13: contexts of use: Contextual Representations and Pretraining		Sentence and Contextual Word Representations (2/12/2019)	http://jalammar.github.io/illustrated-bert/
✨***Transformer***✨	Transformers and Self-Attention For Generative Models			https://jalammar.github.io/illustrated-transformer/ http://nlp.seas.harvard.edu/2018/04/03/attention.html
Dependency Parsing	lecture 5: Linguistic Structure: Dependency Parsing	chapter 8: Part-of-Speech Tagging
Structured Prediction Models			Search-based Structured Prediction (2/19/2019) Reinforcement Learning (2/21/2019) Structured Prediction with Local Independence Assumptions (2/26/2019)
Advanced Learning Techniques			Latent Random Variables (3/5/2019) Adversarial Methods for Text (3/7/2019) Unsupervised and Semi-supervised Learning of Structure (3/28/2019)
✨Models of Knowledge and Context✨	Reference in Language and Coreference Resolution		Models of Dialog (4/2/2019) Document-level Models (4/4/2019) Learning from/for Knowledge Graphs (4/9/2019) Machine Reading w/ Neural Nets (4/16/2019)
Multi-task and Multilingual Learning	Multitask Learning: A general model for NLP?		Multi-task Multi-lingual Learning Models (4/18/2019) Multimodal Models (4/23/2019)