Awesome
nlp journey
1. Books
- Handbook of Graphical Models.
online
- Deep Learning.
online
- Neural Networks and Deep Learning.
online
- Speech and Language Processing.
online
2. Papers
01) Transformer papers
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.
paper
- GPT-2: Language Models are Unsupervised Multitask Learners.
paper
- Transformer-XL: Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context.
paper
- XLNet: Generalized Autoregressive Pretraining for Language Understanding.
paper
- RoBERTa: Robustly Optimized BERT Pretraining Approach.
paper
- DistilBERT: a distilled version of BERT: smaller, faster, cheaper and lighter.
paper
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations.
paper
- T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.
paper
- ELECTRA: pre-training text encoders as discriminators rather than generators.
paper
- GPT3: Language Models are Few-Shot Learners.
paper
02) Models
- LSTM(Long Short-term Memory).
paper
- Sequence to Sequence Learning with Neural Networks.
paper
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation.
paper
- Residual Network(Deep Residual Learning for Image Recognition).
paper
- Dropout(Improving neural networks by preventing co-adaptation of feature detectors).
paper
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.
paper
03) Summaries
- An overview of gradient descent optimization algorithms.
paper
- Analysis Methods in Neural Language Processing: A Survey.
paper
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.
paper
- A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications.
paper
- A Gentle Introduction to Deep Learning for Graphs.
paper
- A Survey on Deep Learning for Named Entity Recognition.
paper
- More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction.
paper
- Deep Learning Based Text Classification: A Comprehensive Review.
paper
- Pre-trained Models for Natural Language Processing: A Survey.
paper
- A Survey on Contextual Embeddings.
paper
- A Survey on Knowledge Graphs: Representation, Acquisition and Applications.
paper
- Knowledge Graphs.
paper
- Pre-trained Models for Natural Language Processing: A Survey.
paper
04) Pre-training
- A Neural Probabilistic Language Model.
paper
- word2vec Parameter Learning Explained.
paper
- Language Models are Unsupervised Multitask Learners.
paper
- An Empirical Study of Smoothing Techniques for Language Modeling.
paper
- Efficient Estimation of Word Representations in Vector Space.
paper
- Distributed Representations of Sentences and Documents.
paper
- Enriching Word Vectors with Subword Information(FastText).
paper
- GloVe: Global Vectors for Word Representation.
online
- ELMo (Deep contextualized word representations).
paper
- Pre-Training with Whole Word Masking for Chinese BERT.
paper
05) Classification
- Bag of Tricks for Efficient Text Classification (FastText).
paper
- Convolutional Neural Networks for Sentence Classification.
paper
- Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification.
paper
06) Text generation
- A Deep Ensemble Model with Slot Alignment for Sequence-to-Sequence Natural Language Generation.
paper
- SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient.
paper
07) Text Similarity
- Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks.
paper
- Learning Text Similarity with Siamese Recurrent Networks.
paper
- A Deep Architecture for Matching Short Texts.
paper
08) QA
- A Question-Focused Multi-Factor Attention Network for Question Answering.
paper
- The Design and Implementation of XiaoIce, an Empathetic Social Chatbot.
paper
- A Knowledge-Grounded Neural Conversation Model.
paper
- Neural Generative Question Answering.
paper
- Sequential Matching Network A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots.
paper
- Modeling Multi-turn Conversation with Deep Utterance Aggregation.
paper
- Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network.
paper
- Deep Reinforcement Learning For Modeling Chit-Chat Dialog With Discrete Attributes.
paper
09) NMT
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation.
paper
- Neural Machine Translation by Jointly Learning to Align and Translate.
paper
- Transformer (Attention Is All You Need).
paper
10) Summary
- Get To The Point: Summarization with Pointer-Generator Networks.
paper
- Deep Recurrent Generative Decoder for Abstractive Text Summarization.
paper
11) Relation extraction
- Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks.
paper
- Neural Relation Extraction with Multi-lingual Attention.
paper
- FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation.
paper
- End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures.
paper
12) Large Language Models
- Training language models to follow instructions with human feedback.
paper
- LLaMA: Open and Efficient Foundation Language Models.
paper
3. Articles
- TRANSFORMERS FROM SCRATCH.
url
- The Illustrated Transformer.
url
- Attention-based-model.
url
- Modern Deep Learning Techniques Applied to Natural Language Processing.
url
- Illustrated Guide to LSTM’s and GRU’s: A step by step explanation
.
url
- Applying word2vec to Recommenders and Advertising.
url
4. Github