Home

Awesome

DeepPavlov 1.0

License Apache 2.0 Python 3.6, 3.7, 3.8, 3.9, 3.10, 3.11 Downloads Static Badge Static Badge

DeepPavlov 1.0 is an open-source NLP framework built on PyTorch and transformers. DeepPavlov 1.0 is created for modular and configuration-driven development of state-of-the-art NLP models and supports a wide range of NLP model applications. DeepPavlov 1.0 is designed for practitioners with limited knowledge of NLP/ML.

Quick Links

nameDescription
⭐️ DemoCheck out our NLP models in the online demo
📚 DocumentationHow to use DeepPavlov 1.0 and its features
🚀 Model ListFind the NLP model you need in the list of available models
🪐 Contribution GuidePlease read the contribution guidelines before making a contribution
🎛 IssuesIf you have an issue with DeepPavlov, please let us know
ForumPlease let us know if you have a problem with DeepPavlov
📦 BlogsRead about our current development
🦙 Extended colab tutorialsCheck out the code tutorials for our models
🌌 Docker HubCheck out the Docker images for rapid deployment
👩‍🏫 FeedbackPlease leave us your feedback to make DeepPavlov better

Installation

  1. DeepPavlov supports Linux, Windows 10+ (through WSL/WSL2), MacOS (Big Sur+) platforms, Python 3.6, 3.7, 3.8, 3.9 and 3.10. Depending on the model used, you may need from 4 to 16 GB RAM.

  2. Create and activate a virtual environment:

    • Linux
    python -m venv env
    source ./env/bin/activate
    
  3. Install the package inside the environment:

    pip install deeppavlov
    

QuickStart

There is a bunch of great pre-trained NLP models in DeepPavlov. Each model is determined by its config file.

List of models is available on the doc page in the deeppavlov.configs (Python):

from deeppavlov import configs

When you're decided on the model (+ config file), there are two ways to train, evaluate and infer it:

GPU requirements

By default, DeepPavlov installs models requirements from PyPI. PyTorch from PyPI could not support your device CUDA capability. To run supported DeepPavlov models on GPU you should have CUDA compatible with used GPU and PyTorch version required by DeepPavlov models. See docs for details. GPU with Pascal or newer architecture and 4+ GB VRAM is recommended.

Command line interface (CLI)

To get predictions from a model interactively through CLI, run

python -m deeppavlov interact <config_path> [-d] [-i]

You can train it in the same simple way:

python -m deeppavlov train <config_path> [-d] [-i]

Dataset will be downloaded regardless of whether there was -d flag or not.

To train on your own data you need to modify dataset reader path in the train config doc. The data format is specified in the corresponding model doc page.

There are even more actions you can perform with configs:

python -m deeppavlov <action> <config_path> [-d] [-i]

Python

To get predictions from a model interactively through Python, run

from deeppavlov import build_model

model = build_model(<config_path>, install=True, download=True)

# get predictions for 'input_text1', 'input_text2'
model(['input_text1', 'input_text2'])

where

You can train it in the same simple way:

from deeppavlov import train_model 

model = train_model(<config_path>, install=True, download=True)

To train on your own data you need to modify dataset reader path in the train config doc. The data format is specified in the corresponding model doc page.

You can also calculate metrics on the dataset specified in your config file:

from deeppavlov import evaluate_model 

model = evaluate_model(<config_path>, install=True, download=True)

DeepPavlov also allows to build a model from components for inference using Python.

License

DeepPavlov is Apache 2.0 - licensed.