Home

Awesome

Table of Contents:

  1. Introduction to Torch's Tensor Library
  2. Computation Graphs and Automatic Differentiation
  3. Deep Learning Building Blocks: Affine maps, non-linearities, and objectives
  4. Optimization and Training
  5. Creating Network Components in Pytorch
  1. Word Embeddings: Encoding Lexical Semantics
  1. Sequence modeling and Long-Short Term Memory Networks
  1. Advanced: Dynamic Toolkits, Dynamic Programming, and the BiLSTM-CRF

What is this tutorial?

I am writing this tutorial because, although there are plenty of other tutorials out there, they all seem to have one of three problems:

Specifically, I am writing this tutorial for a Natural Language Processing class at Georgia Tech, to ease into a problem set I wrote for the class on deep transition parsing. The problem set uses some advanced techniques. The intention of this tutorial is to cover the basics, so that students can focus on the more challenging aspects of the problem set. The aim is to start with the basics and move up to linguistic structure prediction, which I feel is almost completely absent in other Pytorch tutorials. The general deep learning basics have short expositions. Topics more NLP-specific received more in-depth discussions, although I have referred to other sources when I felt a full description would be reinventing the wheel and take up too much space.

Dependency Parsing Problem Set

As mentioned above, here is the problem set that goes through implementing a high-performing dependency parser in Pytorch. I wanted to add a link here since it might be useful, provided you ignore the things that were specific to the class. A few notes:

References:

Exercises:

There are a few exercises in the tutorial, which are either to implement a popular model (CBOW) or augment one of my models. The character-level features exercise especially is very non-trivial, but very useful (I can't quote the exact numbers, but I have run the experiment before and usually the character-level features increase accuracy 2-3%). Since they aren't simple exercises, I will soon implement them myself and add them to the repo.

Suggestions:

Please open a GitHub issue if you find any mistakes or think there is a particular model that would be useful to add.