Home

Awesome

NLP for Hindi

This repository contains State of the Art Language models and Classifier for Hindi language (spoken in Indian sub-continent).

The models trained here have been used in Natural Language Toolkit for Indic Languages (iNLTK)

Dataset

Created as part of this project

  1. Hindi Wikipedia Articles - 172k

  2. Hindi Wikipedia Articles - 55k

  3. Hindi Movie Reviews Dataset

  4. Hindi Text Short Summarization Corpus

  5. Hindi Text Short and Large Summarization Corpus

Open Source Datasets

  1. BBC News Articles : Sentiment analysis corpus for Hindi documents extracted from BBC news website.

  2. IIT Patna Product Reviews : Sentiment analysis corpus for product reviews posted in Hindi.

  3. IIT Patna Movie Reviews : Sentiment analysis corpus for movie reviews posted in Hindi.

Results

Language Model Perplexity (on validation set)

Architecture/DatasetHindi Wikipedia Articles - 172kHindi Wikipedia Articles - 55k
ULMFiT34.0635.87
TransformerXL26.0934.78

Note: Nirant has done previous SOTA work with Hindi Language Model and achieved perplexity of ~46. The scores above aren't directly comparable with his score because his train and validation set were different and they aren't available for reproducibility

Classification Metrics

ULMFiT
DatasetAccuracyMCCNotebook to Reproduce results
BBC News Articles78.7571.61Link
IIT Patna Movie Reviews57.7437.23Link
IIT Patna Product Reviews75.7159.76Link

Visualizations

Word Embeddings
ArchitectureVisualization
ULMFiTEmbeddings projection
TransformerXLEmbeddings projection
Sentence Embeddings
ArchitectureVisualization
ULMFiTEncodings projection

Results of using Transfer Learning + Data Augmentation from iNLTK

On using complete training set (with Transfer learning)
DatasetDataset size (train, valid, test)AccuracyMCCNotebook to Reproduce results
IIT Patna Movie Reviews(2480, 310, 310)57.7437.23Link
On using 20% of training set (with Transfer learning)
DatasetDataset size (train, valid, test)AccuracyMCCNotebook to Reproduce results
IIT Patna Movie Reviews(496, 310, 310)47.7420.50Link
On using 20% of training set (with Transfer learning + Data Augmentation)
DatasetDataset size (train, valid, test)AccuracyMCCNotebook to Reproduce results
IIT Patna Movie Reviews(496, 310, 310)56.1334.39Link

Pretrained Models

Language Models

Download pretrained Language Models of ULMFiT, TransformerXL trained on Hindi Wikipedia Articles - 172k and Hindi Wikipedia Articles - 55k from here

Tokenizer

Unsupervised training using Google's sentencepiece

Download the trained model and vocabulary from here